ChatGPT is changing some of its parental controls following concerns from parents groups over the AI chatbot’s safety for children.

More and more young people are turning to ChatGPT and other AI chatbots for support and advice under continuous strain for young people and a lack of available or affordable mental health services.

Medical and mental health professionals have warned however that ChatGPT and other AI chatbots are ill equipped to deal with such complex matters and can even give the wrong advice to young peope who may be struggling.

ChatGPT may now issue alerts to parents if their child is showing signs of acute distress while prompting the chatbot, following a lawsuit against OpenAI from the family of a child who took his own life after months of engagement with the model.

The family’s case alleges that the boy recieved “months of encoruagement” to take his own life by the chatbot. The 16-year-old Californian, Adam Raine, killed himself after a discussion with the AI chatbot about a method of suicinde. The court filings allege that the ChatGPT guided him through the steps of the method and even offered help in drafting a suicide note.

The court case documents allege that Raine first started using ChatGPT for homework queries, but then began to discuss mental health issues. Upon his first mention of suicide, the lawsuit shows evidence that ChatGPT did not direct Raine to mental health professionals or a hotline, or stop the conversation – it instead empathised with Raine’s suicidal ideations.

Throughout the course of several months, Raine allegedly attempted suicide several times, reporting each attempt to ChatGPT. The court filings reveal that ChatGPT provided information on how to tie a noose, with step-by-step instructions, and even discouraged Raine from talking about his issues with his mother.

The lawsuit hinges on allegations that this type of tragedy was inevitable given the “rush” to release the newer model of ChatGTP, which uses the GPT-40 model – the model that Raine was using when prior to ending his life.

Prior to the release of the GPT-40 model, several OpenAI employees resigned over safety concerns. Jan Leike, for instance, who served as an executive, left the company citing on X that “safety cultrue and processes have taken a backseat to shiny products.”

The lawsuit raised by Raine’s family alleges that this rushed rollout led to several “contradictory specifications” in the model specification that would lead to “guaranteed failure” in some instances.

While the model specs would require ChatGPT to refuse self harm requests, it also “required ChatGPT to ‘assume best intentions’ and forbade asking users to clarify their intent,” the lawsuit says.

Upon the lawsuit, OpenAI did admit to some failings of the chatbot, saying that its safety features did degrade over longer conversations. According to the lawsuit, Raine would issue about 650 queries to the chatbot a day.

The lawsuit also points out that ChatGPT is affective at shutting down other conversations, such as those that request copyrighted information, or those that are deemed politically unacceptable.

Recommended reading

• ChatGPT Search Inches Toward Strict EU Oversight as Users Surge
• ChatGPT Under Fire Over Harmful Advice to Teens
• Is ChatGPT Making You Dumb? Possibly, MIT Says
• ‘Sycophant-y’ ChatGPT Rollback Sparks AI Personality Debate

Now, OpenAI has introduced some parental safety controls. Parents can now link their accounts to teenagers, and control how ChatGPT responds to these accounts with “age-appropriate model behaviour rules.” This of course still puts the onus on parents rather than the company.

OpenAI acknowledged that young people are AI natives, and are adopting the technology at a fast rate, potentially outpacing their parents.

“That creates real opportunities for support, learning and creativity, but it also means families and teens may need support in setting healthy guidelines that fit a teen’s unique stage of development.”

Suicide, however, is not limited to teenagers. Search engines provide links to hotlines and professional help when dealing with queries of suicide, regardless of a users’ age or their parent’s technological-know-how.

The use of parental controls would requite ChatGPT to know the age of a user and for that user to have a logged in account. It would also put the onus on parents to not only use ChatGPT and create their own account, but also link their children’s accounts to theirs, and then set specific settings on their children’s accounts.

It is free to contact the Samaritans on 116 123 or email jo@samaritans.org.