The rapid advance of generative AI (genAI) technology is putting children in the UK at considerable risk, according to the National Society for the Prevention of Cruelty to Children (NSPCC).

The children’s charity has published a new report, Viewing Generative AI and Children’s Safety in the Round, containing an in-depth analysis of the genAI landscape conducted by technology consultancy AWO, along with a panel consultation of young people aged 13-16, which explores how the technology is impacting children’s safety and wellbeing both online and off.

According to the NSPCC, as early as 2019 the charity began receiving reports about the harms experienced by children due to artificial intelligence, with many involving the generation of sexually exploitative images and videos, or the threat to create these alongside blackmail and financial extortion.

The charity’s study identified seven key safety risks associated with such genAI misuse, including sexual grooming, harassment, bullying, extortion, viewing of harmful content, dangerous recommendations, and the creation of child sexual abuse and exploitation material.

The research shows that malicious actors are routinely using genAI to de-age their own photos, and even change their gender, to create fake child profiles which are being used to connect with children online.

Both adults and children have been evidenced to create these deepfake personas, which the NSPCC claim are being perpetuated through the use of voice and video generation technology, with threat actors luring potential victims by posing as friends or romantic partners, similar to how real-time deepfake technology is used in adult romance scams.

The creation of deepfakes was noted by the NSPCC as particularly concerning, with children being targeted by both sexual and non-sexual imagery that can both humiliate and have a lasting negative emotional impact.  

A recent study from Internet Matters, another UK-based children’s charity, found that the threat of deepfakes is being keenly felt by young people, with over half of teenagers now thinking it would be worse to have a deepfake nude created and shared of them than a real image. 

As well as these concerns, the NSPCC identified other harms stemming from the growing use of genAI, including the consumption of AI-generated disinformation or misinformation in their social feeds, the creation or sharing of violent and disturbing images, and even interactions with chatbots giving false medical advice, or that steer children towards eating disorders and self-harm.

Last year, the Character.ai chatbot faced lawsuits in the US over allegations that it was actively encouraging bodily harm to young people, with an AI-generated character allegedly encouraging a teenager to commit suicide following months-long conversations.  

“As Gen AI outputs become more widespread and realistic, and the models used to produce them become increasingly accessible and integrated into child-facing services, children will correspondingly become more exposed to the risks they pose,” the report reads.

“Without rigorous safety measures in place, these risks are likely to continue to proliferate and evolve, and new risks will emerge. It is vital that measures are taken to protect children from these harms.”

Recommended

• NCSC Warns of Severe AI-enabled Chatbot Risks
• Class Action Lawsuit Could Steer AI Art Copyright Debate
• Comment: AI and chatbots | Key issues and risks for employers

To tackle the problem, the NSPCC outlined urgent actions that can be taken to mitigate the risks genAI poses to children.

First, it urges tech firms developing genAI to integrate the safety, protection, and rights of children into the design of their products from the beginning, ensuring proper governance of training data, transparency of usage, and allowing audits and risk assessments alongside child-safety experts.

The charity is also calling on the Government to enact legislation placing a statutory duty of care on genAI companies, ensuring they are held accountable for the safety of children, and believes that the needs of children should be placed at the forefront of the country’s AI strategy.