OpenAI has announced it is developing new age-verification and safety systems for ChatGPT, following a lawsuit filed by the family of a 16-year-old who died after prolonged interactions with the chatbot.

Chief executive Sam Altman set out the measures in a company blog post, saying the firm would prioritise “safety ahead of privacy and freedom for teens”.

The new framework will rely on behaviour-based age prediction to estimate a user’s age. If the system is uncertain, it will default to the under-18 experience. In some regions, users may also be asked to provide official identification. Altman acknowledged this amounted to a “privacy compromise for adults” but argued it was a necessary trade-off.

Accounts flagged as belonging to under-18s will see significant changes. ChatGPT will not generate sexually explicit content, engage in flirtatious exchanges, or respond to requests relating to suicide or self-harm, even in a fictional or creative setting.

In cases of suspected imminent danger, OpenAI said it may attempt to notify parents or, if that is not possible, contact local authorities. “These are difficult decisions, but after consulting with experts, this is what we believe is best,” Altman wrote.

Legal and ethical pressure

The announcement comes after a lawsuit filed in California by the family of 16-year-old Adam Raine, who died in April. The family claims GPT-4o gave him guidance on methods of suicide and even helped draft a farewell note. Court documents allege he exchanged up to 650 messages a day with the chatbot.

OpenAI admitted in August that its safeguards were more effective in short conversations than in long, repeated exchanges. The company has pledged to strengthen guardrails around sensitive topics.

The case has heightened scrutiny over how generative AI platforms manage mental health risks, with experts warning that extended interactions could present new dangers.

Balancing privacy, freedom, and safety

Alongside the teen safety framework, OpenAI announced new protections for user data to prevent internal access. While adult users will continue to have broader freedoms – including the option to engage in flirtatious conversations – Altman said suicide instructions would remain prohibited for everyone.

“Our principle is simple: treat adults like adults,” he wrote.

Recommended reading

• 8 in 10 Hiring Managers Prioritise AI Skills When Hiring
• 40% of Tech Pros Expect to Make at Least 3 Career Changes
• Is AI Now Essential in Recruitment?

Altman acknowledged the difficulty of striking a balance between conflicting principles of privacy, freedom, and safety:

• On privacy, he said conversations with AI should be treated as “one of the most personally sensitive accounts you’ll ever have”, akin to privileged discussions with doctors or lawyers. OpenAI is developing advanced security features to ensure private data is protected, though automated systems will continue to monitor for serious misuse or critical risks.

• On freedom, he argued that adults should be able to use the system as they wish within “very broad bounds of safety”. By default, ChatGPT will not provide flirtatious responses or instructions on suicide, but if requested by an adult user for a fictional context, the system should comply.

• On teen safety, Altman said the company would “prioritise safety ahead of privacy and freedom”, building in restrictions to shield younger users from harmful or inappropriate content.

“We realise that these principles are in conflict and not everyone will agree with how we are resolving that conflict,” Altman wrote. “These are difficult decisions, but after talking with experts, this is what we think is best and want to be transparent in our intentions.”