Tech companies should take action to stop illegal content from going viral, prevent terrorism content and explicit deepfakes at source and stop children from being groomed through livestreams, under new proposals from Ofcom.

The new measures continue Ofcom’s implementation of the Online Safety Act, building on the regulator’s illegal harms and children’s safety codes of practice, which are already in place and being enforced..

Stopping illegal content going viral

If illegal content spreads rapidly online, it can lead to severe and widespread harm, especially during a crisis, like the violent riots that followed the Southport murders last year, or if a terrorist attack is livestreamed. Recommender systems can exacerbate this.

To prevent this from happening, Ofcom has stated that platforms should have protocols in place to respond to spikes in illegal content during a crisis, and should not recommend material to users where there are indicators it might be illegal, unless and until it has been reviewed.

If a site or app allows livestreaming, it should have a system which makes it clear to them when a user reports a livestream where there is a risk of imminent physical harm, and have human moderators available at all times to review content and take action in real time.

Tackling harms at source

Huge volumes of content appear online every day, so providers need to make effective use of technology to make their sites and apps safer by design and prevent illegal material from reaching users. Ofcom says that they should use a technique called hash matching to detect terrorism content and intimate images that are shared without consent, such as explicit deepfakes.

Ofcom also propose that some services should assess the role automated tools can play in detecting content including previously undetected child sexual abuse material, content promoting suicide and self-harm and fraudulent content – and use them where they are available and effective.

Further protections for children

Livestreaming has many benefits – for gaming, showcasing talents, citizen-journalism or sharing real-world experiences. However, it also presents risks with children risk being groomed, coerced into performing sexual acts, or encouraged into acts of self-harm and suicide while livestreaming.

Ofcom is proposing that sites and apps should prevent people from posting comments or reactions or sending gifts to children’s livestreams, and they should prevent people from recording children’s livestreams.

Under the regulator’s existing codes, providers should already be taking steps to protect children from grooming. Now that Ofcom have published guidance effective age assurance, platforms should use robust age checks to underpin the measures they take to protect children from grooming and harms associated with livestreaming.

Recommended reading

• Ofcom Investigates Two Porn Sites For Alleged Age Check Failings
• Children’s Commissioner Hits Out at AI DeepFake Images
• Notorious Deepfake Porn Site Mr. Deepfakes Shuts Down for Good

Ofcom also expect companies to ban users who share child sexual exploitation and abuse material.

Oliver Griffiths, Ofcom’s Online Safety Group Director, said: “Important online safety rules are already in force and change is happening. We’re holding platforms to account and launching swift enforcement action where we have concerns.

“But technology and harms are constantly evolving, and we’re always looking at how we can make life safer online. So today we’re putting forward proposals for more protections that we want to see tech firms roll out.”

Ofcom has also published an update on our other areas of work to implement the Online Safety Act.