Chatacter.AI, the AI chat bot firm, is banning under 18s from certain features after serious criticisms and lawsuits against the firm for its affect on teenagers’ psyche and mental health.

The AI firm allows users to create fictional AI characters that they can chat with, create content with, and chat with other AI characters. Characters can be based on real people, existing fictional characters, or can be user generated.

With the new rules, from 25 November, teenagers will no longer be able to talk to characters, but can still create video content with AI characters on the site.

Between now and then, Chatacter.AI says it will work to “build an under-18 experience that still gives our teen users ways to be creative…” Chats or under 18s will also be limited rather than open-ended.

Prior to the rule change, Chatacter.AI faces a number of lawsuits in the US from concerned parents, including parents of a teenager who committed suicide after having conversations for months with an AI character on the platform.

Following the death of the 14-year-old and the lawsuit from his mother, Character.AI introduced Parental Insights, which provided guardians more insight into how their children are using the platform.

However, added pressure from children’s rights groups and internet safety advocates, as well as threats of regulatory scrutiny, has primed the firm to take further action to protect teens.

Recommended

• AI Chatbot Allegedly Encouraged Teen to Kill Parents
• ChatGPT Under Fire Over Harmful Advice to Teens
• NCSC Warns of Severe AI-enabled Chatbot Risks
• Comment: AI and chatbots | Key issues and risks for employers

The firm will introduce user age detection, which it developed in house and will combine with Persona, a third-party firm.

Character.AI will also create an AI safety lab to research how forms of AI entertainment can be safe.

“We’re making these changes to our under-18 platform in light of the evolving landscape around AI and teens,” Chatacter.AI said.

“We have seen recent news reports raising questions, and have received questions from regulators, about the content teens may encounter when chatting with AI and about how open-ended AI chat in general might affect teens, even when content controls work perfectly.”