Ofcom has launched a formal investigation into X to determine whether the platform has complied with its legal duties under the UK’s Online Safety Act, following reports that its Grok AI chatbot has been used to generate and share sexualised imagery.

The UK’s independent online safety regulator said it has opened the investigation to assess whether X has taken appropriate steps to protect people in the UK from content that is illegal under UK law.

The probe follows reports that the Grok account on X was used to create and distribute undressed/sexualised images of people without their consent, which may amount to intimate image abuse or pornography, as well as sexualised images of children that may constitute child sexual abuse material.

Ofcom said it contacted X urgently on 5 January and set a deadline of Friday 9 January for the company to explain what measures it had taken to meet its obligations to protect UK users. The company responded by the deadline, after which Ofcom carried out an expedited assessment of the available evidence.

Following that assessment, Ofcom stated that it decided to open a formal investigation. The inquiry will examine whether X has failed to meet specific requirements under the Online Safety Act, including whether it has properly assessed the risk of UK users encountering illegal content and carried out updated risk assessments before making significant changes to its service.

Beyond this, Ofcom will also assess whether X has taken appropriate steps to prevent people in the UK from seeing illegal content, such as non-consensual intimate images and child sexual abuse material, and whether illegal content is removed as fast at it can be once the platform becomes aware of it.

The investigation will also consider whether X has had regard to protecting users from breaches of privacy law, whether it has adequately assessed the risks its service poses to children in the UK, and whether it has used effective age assurance measures to prevent UK children from accessing pornography.

Ofcom stressed that the legal responsibility for determining whether content breaches UK law rests with platforms themselves.

While companies can use Ofcom’s Illegal Content Judgements Guidance when making such decisions, the regulator said it does not act as a censor and does not direct platforms to remove specific posts or accounts. Instead, its role is to judge whether services have taken appropriate steps to protect people in the UK from illegal content and to protect children from other forms of harmful material.

The investigation will follow the process set out in the Online Safety Act. Ofcom’s first step is to gather and analyse evidence to determine whether a breach has occurred. If the regulator considers that a compliance failure has taken place, it will issue a provisional decision to the company. X would then be given the opportunity to respond fully to Ofcom’s findings before a final decision is made.

If Ofcom concludes that X has broken the law, it has the power to require the platform to take specific actions to come into compliance or to remedy harm caused by any breach. The regulator can also impose financial penalties of up to £18 million or 10% of qualifying worldwide revenue, whichever is greater.

In the most serious cases of ongoing non-compliance, Ofcom can apply to a court for business disruption measures. These could include court orders requiring payment providers or advertisers to withdraw their services from a platform, or requiring internet service providers to block access to a site in the UK.

Such measures can only be imposed where a court considers them appropriate and proportionate to prevent significant harm to individuals in the UK.

Recommended

• The Twitter Blue Tick is Back – With a Higher Price Tag for Some
• X Under Fire Over ‘Staggering Amplification of Hate’ During UK Riots
• Big Tech Faces EU Digital Services Act Reckoning

An Ofcom spokesperson said: “Reports of Grok being used to create and share illegal non-consensual intimate images and child sexual abuse material on X have been deeply concerning. Platforms must protect people in the UK from content that’s illegal in the UK, and we won’t hesitate to investigate where we suspect companies are failing in their duties, especially where there’s a risk of harm to children.

“We’ll progress this investigation as a matter of the highest priority, while ensuring we follow due process. As the UK’s independent online safety enforcement agency, it’s important we make sure our investigations are legally robust and fairly decided.”