The Molly Rose Foundation (MRF) has released new research that reveals children are being put at risk by the failure of major social networks to detect and remove dangerous suicide and self-harm content.

Analysis of over 12 million content moderation decisions by six major tech platforms shows that over 95% of suicide and self-harm posts are being detected by just two major platforms, Pinterest and TikTok.

The report found that Instagram and Facebook are each responsible for only 1% of all suicide and self-harm content detected by major platforms. X, formerly known as Twitter, is responsible for just 1 in 700 content decisions.

MRF is warning that Ofcom’s proposed regulation doesn’t go far enough to address such clear systemic failures. Chair Ian Russell has urged the government to commit to a new Online Safety Act that can strengthen regulation and ‘finish the job.’

The research shows that the response of many tech firms to suicide and self-harm content is ‘inconsistent, uneven and unfit for purpose.’

MRF analysed publicly available records of over 12 million content moderation decisions taken by six sites: Facebook, Instagram, Pinterest, Snapchat, TikTok and X. Under the EU’s Digital Services Act, these platforms must publish records every time they detect and take action on an item of suicide and self-harm content.

Social media platforms are routinely failing to detect harmful content on the highest-risk parts of their services. For example, only 1 in 50 suicide and self-harm posts detected by Instagram were videos, despite its short-form video product Reels now accounting for half of all time spent on the app.

Most major services fail to do enough to enforce their own rules, according to the report: for example, while TikTok detected almost 3 million items of suicide and self-harm content (2,892,658 decisions), it suspended only two accounts.

The foundation also found no evidence that Meta is implementing high-profile commitments it made to restrict harmful suicide and self-harm content from children’s feeds. Despite promising to restrict harmful content in January 2024, shortly before Mark Zuckerberg gave evidence to a US Senate hearing, Meta has so far failed to restrict a single item of content for teens.

Ian Russell, chair of the Molly Rose Foundation, said: “Almost seven years after Molly died, it’s shocking to see most major tech companies continue to sit on their hands and choose inaction over saving young lives.

“As the last few weeks have shown, it’s abundantly clear that much more ambitious regulation is required. That’s why it’s time for the new Government to finish the job and commit to a strengthened Online Safety Act.

Recommended

• Twitter Testing Encrypted Messaging Feature
• Online Safety Bill: Privacy Experts Challenge Encryption Clause
• Online Safety Bill: Where is Encryption Now?

“Parents across the country will be rightly appalled that the likes of Instagram and Facebook promise warm words but continue to expose children to inherently preventable harm. No ifs, no buts, it’s clear that assertive action is required.”

Last year, MRF set out analysis about the amount of suicide and self-harm content freely available on major social media services. The research found that almost half of the most engaged-with posts on both TikTok and Instagram, which were posted using well-known suicide and self-harm hashtags, were likely to be harmful to children.

Earlier this year, Ofcom consulted on its proposals to protect children from the risks of harmful content. MRF is concerned that the proposals lack much-needed ambition, and that the regulator’s approach is unlikely to be effective enough to tackle the scale and complexity of harmful suicide and self-harm material.

Last month, MRF commissioned polling that shows strong support for a new Online Safety Act. Four in five UK adults and 84% of parents support a new Online Safety Act. Among parents who support new legislation, 91% per cent want this to be introduced in the first two years of the Parliament.