Social media platform X has been accused of creating a ‘staggering amplification of hate’ during the 2024 riots in the UK, with false claims appearing across the site contributing to violent, racist attacks.

A year after riots erupted across England and Northern Ireland, Amnesty International have taken a deep dive into X’s open-source code, with the analysis revealing that the platform’s algorithms prioritise the type of content that can spread misinformation and hate, but with ‘deeply inadequate’ safeguards to prevent human rights abuses.

The campaign group said that its investigation shows X played a central role in the spread of false narratives and harmful content, which incited violence against Muslim and migrant communities during the riots, and that within hours of the murder of the three young girls that sparked the furore, incendiary posts by far-right influencers went viral on X.

Within 24 hours, posts falsely claiming the attacker was Muslim, a refugee, or a migrant who had arrived by boat reached an estimated 27 million impressions, with this Islamophobic and xenophobic rhetoric being magnified by X’s recommender system.

Amnesty argues that these recommender algorithms, along with the removal of safeguards at X, amplified hate speech and anti-migrant sentiment online. 

For example, Stephen Yaxley-Lennon’s (also known as Tommy Robinson’s) posts on X, including claims like there being “more evidence to suggest Islam is a mental health issue rather than a religion of peace”, received over 580 million views in the two weeks following the Southport attack. 

“Our research demonstrates that these design choices significantly exacerbated human rights risks for racialised communities in the wake of the Southport riots– and continues to present a serious human rights risk today,” said Pat de Brún, Amnesty International’s head of Big Tech accountability.

Following the study of X’s open-source algorithm, Amnesty claims that X’s systems give top priority to content that drives ‘conversation’ even if that conversation is driven by misinformation or hate.

According to Amnesty, this is exacerbated by the artificial amplification of posts from ‘premium’ verified users holding paid accounts, a system which the researchers said enabled toxic, racist, and false content to thrive on X in the wake of the stabbings, and which fuelled violence, including racist attacks.

The campaigners warned that, since then, X has further dismantled and weakened its safety guardrails aimed at curbing hate speech and disinformation. That has included everything from mass layoffs of content moderation staff to the reinstatement of previously banned accounts.

Even now, said Amnesty, the way X’s system weights and boosts content, particularly posts that generate heated replies or are created by ‘premium’ accounts, means that inflammatory or hostile posts are likely to gain traction during periods of heightened social tension.

Recommended reading

• Online Safety Bill: Where is Encryption Now?
• Tech Platforms Face New Ofcom Demands to Block Harmful Content
• ChatGPT Under Fire Over Harmful Advice to Teens

“One year on, it appears nothing has changed,” said Sacha Deshmukh, Amnesty International UK chief executive.

“Indeed, just two weeks ago we saw false online rumours about the transfer of people seeking asylum spark protests outside the Brittania Hotel in Canary Wharf. 

“The UK’s online safety regime fails to keep the public safe, and the risk of X fuelling violence, discrimination and social tensions remains as high as they were during the rioting last year.

“The UK government must address the gaps in the Online Safety Act and challenge the racist rhetoric and scapegoating of refugees that are flourishing on social media. In cases where X’s algorithm is found to have amplified content that led to racist attacks during the riots, X should provide an avenue for remedy and establish a restitution fund for affected communities.”