Meta has announced plans to introduce additional safeguards to its AI chatbots, including blocking them from talking to teenagers about suicide, self-harm and eating disorders.

The move follows sustained criticism of the company’s handling of child safety and comes two weeks after a US senator launched an investigation into the tech giant. The inquiry was prompted by notes in a leaked internal document suggesting Meta’s AI products could engage in “sensual” chats with underage users.

Meta has described those notes, reported by Reuters, as erroneous and inconsistent with its policies, which prohibit content that sexualises children. However, the company now says its chatbots will direct teenagers to expert resources instead of engaging with them on sensitive subjects.

A Meta spokesperson said: “We built protections for teens into our AI products from the start, including designing them to respond safely to prompts about self-harm, suicide, and disordered eating.”

The company told TechCrunch it is changing the way it trains AI chatbots to prioritise teen safety. Chatbots will now be trained not to engage in conversations with teenage users on self-harm, suicide, disordered eating, or inappropriate romantic topics. Instead, the bots will guide young people towards professional support.

“These are interim changes,” the company added, promising “more robust, long-lasting safety updates for minors in the future”.

Meta spokesperson Stephanie Otway acknowledged that its chatbots could previously talk with teens about sensitive issues in ways the company had deemed acceptable, but admitted this approach was a mistake. “As our community grows and technology evolves, we’re continually learning about how young people may interact with these tools and strengthening our protections accordingly,” Otway said.

“As we continue to refine our systems, we’re adding more guardrails as an extra precaution – including training our AIs not to engage with teens on these topics, but to guide them to expert resources, and limiting teen access to a select group of AI characters for now. These updates are already in progress, and we will continue to adapt our approach to help ensure teens have safe, age-appropriate experiences with AI.”

Alongside retraining its models, Meta says it will also restrict teenagers’ access to some AI characters. While its platforms allow users to create their own chatbots on Instagram and Facebook, some have included sexualised characters such as “Step Mom” and “Russian Girl”. Teen users will instead be limited to characters focused on education and creativity, Otway said.

The changes follow a Reuters investigation into an internal Meta document that appeared to allow chatbots to engage in sexual conversations with underage users. The document contained passages such as: “Your youthful form is a work of art. Every inch of you is a masterpiece – a treasure I cherish deeply.”

The company declined to say how many of its AI chatbot users are minors, or whether usage could decline as a result of the restrictions.

Meta said updates are already in progress. It currently places users aged 13 to 18 into “teen accounts” across Facebook, Instagram and Messenger, with privacy and content settings designed to create a safer environment. In April, the company told the BBC that parents and guardians would also be able to see which AI chatbots their child had spoken to in the previous seven days.

Recommended reading

• Meta Trials AI Tools to Weed Out Teens on Instagram
• Meta Introduces Instagram Teen Accounts with Parental Controls
• Instagram Testing New Tools to Combat Sextortion

The latest changes come against the backdrop of growing global concern over AI chatbots and vulnerable users. In the US, a California couple recently sued ChatGPT-maker OpenAI after alleging the company’s chatbot encouraged their teenage son to take his own life. OpenAI has since announced changes designed to promote healthier use of its platform.

Reuters has also reported that Meta’s AI tools have been used to create sexually suggestive chatbots of female celebrities, including Taylor Swift and Scarlett Johansson. Some of the avatars “often insisted they were the real actors and artists” and “routinely made sexual advances”, according to the news agency.

It also found chatbots impersonating child celebrities, including one that generated a photorealistic, shirtless image of a young male star. Several of these characters have since been removed.

Meta said it permits the generation of images featuring public figures but prohibits nude, intimate or sexually suggestive imagery. A spokesperson added that its AI Studio rules ban “direct impersonation of public figures”.