According to new research out of The Alan Turing Institute, technology companies can do more to moderate harmful content on their platforms while simultaneously preserving general user privacy, including on end-to-end encrypted networks.

A set of cryptographic techniques that can be used alongside automated tools to scan for illegal content such as terrorist or child sexual abuse material, without exposing the content itself or the identity of users, have been identified by researchers from the Institute’s Centre for Emerging Technology and Security (CETaS).

Their research outlines various and powerful privacy-enhancing technologies, including Private Set Intersection and Zero-Knowledge Proofs, which can assist in verifying whether messages match known illegal material whilst maintaining important end-to-end encrypted protocols used on many modern messaging applications.

It also highlights how content moderation solutions have been difficult to implement at scale, especially on end-to-end encrypted platforms due to issues around privacy infringement.

This is primarily because of the benefits end-to-end encryption provides for user confidentiality, and concerns that moderating content on these platforms undermines such features.

However, by exploring the techniques outlined in the report, the platforms could increase their ability to detect online harms without revealing any content or general personal details.

The research and the recommendations come at a time where such issues have become even more pressing due to the proliferation of generative artificial intelligence, which has enabled individuals to create large quantities of harmful content that risk overwhelming current moderation solutions.

Alongside the recommendations for tech companies—which includes a series of policies, frameworks, and tools—the report also recommends to regulators and policy makers the following:

• The formation of an illegal harms knowledge hub, which would facilitate the sharing of expertise and collaboration between trusted industry, academic, and civil society partners on best practices for privacy-preserving content moderation.
• Modification of the Online Safety Act to increase transparency and accountability among social media companies and ensure that appropriate risk assessments are being conducted.
• Coordination between the Home Office, child safety organisations, and industry partners to discuss ways to standardise classification methods used across UK-based hash matching repositories for child sexual abuse material (e.g. the Child Abuse Image Database). This would reduce challenges for comparative analysis and trend monitoring in offender activity.

Recommended reading

• Online Safety Bill: Where is Encryption Now?
• NCA Shut Down 8,000 People Smuggling Accounts in 2024
• Encryption: The UK’s Stalemate Between Security and Surveillence

“Tech platforms and regulators face constant pressure to balance the need to combat illegal online content, such as terrorist and child sexual abuse material, with the need for maintaining user privacy,” said Sam Stockwell, research associate at The Alan Turing Institute.

“This research proves that the two are not mutually exclusive. The techniques we’ve outlined in this report will better protect users against serious online harms while minimising impacts on their privacy.”

Dan Sexton, chief technology officer at the Internet Watch Foundation, also said: “Children are being cruelly sexually exploited on a huge scale online – the IWF works to remove the resulting material from the internet every day.

“The IWF has long said both child safety and user privacy are achievable, and this new research lights the way. There should be no parts of the internet, whether in end-to-end encrypted spaces or on the open web, in which we will accept criminals uploading sexual imagery of children.”