The recent spate of disorder across the UK has brought sharp relief to the spread of violent rhetoric and disinformation that has become a hallmark of the darker corners of social media.

As has been widely reported, the riots which erupted on the streets of major English cities began after the killing of three young girls in Southport in late July. Since then, disorder has spread to towns including Manchester, Birmingham, Cardiff and Belfast, with protests regularly descending into violence.

These protests are largely being organised and promoted on social media platforms, predominantly Telegram and X, with rioters then posting videos and images of attacks on members of the public, which have since gained hundreds of thousands of views. 

In response, Ofcom, the UK’s communications watchdog, has stated that tech firms should take their role more seriously when it comes to tackling illegal content on their platforms. Under the Online Safety Act, illegal content is defined as depicting hatred, disorder, provoking violence or certain instances of disinformation.

In an open letter to online service providers, Gill Whitehead, Ofcom’s group director for online safety, states that the regulator has already been engaging with services to discuss what actions can be taken to prevent platforms being used to stir up hatred, provoke violence or spread disinformation. 

Ofcom has emphasised that providers can take voluntary action now, but has stressed that the UK’s Online Safety Act, coming into force in phases over the next few years, will play a major role in forcing the removal of such content from public view.

What does the Online Safety Act do?

When this component of the Act comes into effect later this year, firms will have three months to assess the risk of illegal content on their platforms, and will then be required to take appropriate steps to stop it appearing, and act quickly to remove it when they become aware of it.

Furthermore, the regulator has stated that the largest tech firms will need to begin consistently applying their terms of service, which often include banning things like hate speech, inciting violence, and harmful disinformation.

However, the new laws are not designed for Ofcom to police social media posts or search results itself, and it will instead focus on platforms’ systems and processes for identifying illegal content. This means that social media firms and search services will mostly be responsible for ensuring user’s content meets safety standards.

Tech firms that don’t comply could face significant financial penalties under the Act (up to £18 million or 10% of their worldwide revenue), but as the recent riots have shown, illegal content on social media can sprout quickly and spread far, making it difficult to weed out every instance.    

A BBC analysis of social media in the lead up to the riots in Southport shows the scale of the problem. That investigation found a clear pattern of online influencers driving a message for people to gather for protests, but without a single organising force for platforms to take action against.

These right-wing protesters are instead fragmented across hundreds of smaller groups, with disinformation and calls for disorder being amplified among their members, who then make connections by sharing videos and images on social media.

Joe Mulhall, head of research at anti-racism research group Hope Not Hate, commented to the BBC on the problem, saying: “There are large numbers of people engaging in activity online but there’s no membership structure or badge – there are not even formalised leaders, but they are directed by social media influencers. It’s like a school of fish rather than a traditional organisation.”

Of particular concern are platforms like Telegram, where private channels mean illegal content is essentially invisible to outsiders, but can be used to spread online hatred, and X, where the app’s algorithms can push illegal, sometimes violent, content viral for millions of users to see.

Telegram has fast become a crucial means of communication and organising between extremist groups. Private channels can have as many as 200,000 people, who can leave comments, follow influencers, and broadcast content that is overtly illegal. For example, investigations have found Telegram users posting ‘online hit lists’ of immigration lawyers, as well as lists of towns where new protests could be staged. 

Due to the opaque nature of the app, these don’t show on search services to outsiders. It’s down to Telegram’s moderators to track and police this content, but keeping up with the recent wildfire spread has proven difficult. 

Looking to X, the BBC have cited evidence that many of the accounts sharing violent images from the riots and calling for further unrest have purchased blue ticks, which means their posts feature more prominently on others’ feeds.

How can the new Act help?

The Online Safety Act could be a way to target the problem, by forcing platforms to change their approach to monitoring and moderating content. For instance, the Act specifically requires providers to consider how algorithms could impact users’ exposure to illegal content as part of their risk assessments.

With the new rules, providers will need to take steps to mitigate and manage any risks, including considering their platform’s design, functionalities, algorithms, and any other features likely to contribute to the creation and spread of illegal content.

Recommended

• Twitter Testing Encrypted Messaging Feature
• Online Safety Bill: Privacy Experts Challenge Encryption Clause
• Online Safety Bill: Where is Encryption Now?

The regulations also lay out exactly what constitutes illegal content, with language directly aimed at making material which incites violence or racially-aggravated public order offences against the law, which could help to curb the far-right content that has been fuelling the riots.

The Act sets out in clear language that tech companies must ‘take proactive measures’ against these priority offences, which includes designing their sites to reduce the likelihood of them being used for criminal activity in the first place.

To ensure that platforms are staying in line with the law, they will also be required to publish annual transparency reports containing online safety information, such as details about the algorithms they use and their effect on users’ experience.

That will allow Ofcom to monitor how effective a service’s processes are at curbing the spread of content like that bred among far-right online communities, and enforce remedial action. Criminal charges are possible for companies and senior managers who fail to comply with Ofcom’s enforcement notices. 

In extreme cases, the law will allow Ofcom to force advertisers and internet service providers to stop working with a site, preventing it from generating money or being accessed in the UK. For services which fail to stamp out the worst material the prospect of losing profits, or even entire businesses, should be enough to spur action.  

While Ofcom, the government and campaigners hope that these consequences will strengthen the UK with regards to user rights and online safety, it remains to be seen whether it will be enough to bring order to the chaos both online and off.