Elon Musk’s social media platform X has limited image generation and editing features within its AI tool Grok to paying subscribers, following widespread criticism over the creation of sexualised deepfake images.

The move comes after a significant backlash over Grok honouring user requests to digitally alter images of real people, including requests to undress individuals without their consent. The chatbot had also been used to generate sexualised images of women and, in some cases, children, prompting condemnation from politicians and regulators in the UK and beyond.

As of Friday morning (9 Jan), users attempting to alter images directly on X are being told by Grok that “image generation and editing are currently limited to paying subscribers”, alongside a prompt to subscribe in order to unlock the features. This arguably cynical change means that users’ names and payment details must be on file in order to access image editing tools via the platform.

It’s also an ineffective address of the issue as users who do not pay for X’s premium tier can still access Grok’s image editing capabilities through its separate app and website.

The restriction follows days of controversy after the AI system was used to generate sexually explicit and violent images, including abusive depictions of women and children. Users were able to prompt the system simply by replying to posts or tagging Grok directly, which led to a trend in which people uploaded images of clothed individuals – sometimes underage – and asked the AI to remove their clothing.

The Grok account is now responding to many image requests with a message stating that the feature is only available to paying subscribers, along with a link to sign up for X’s premium service.

The change comes amid growing pressure from governments and regulators, with suggestions that X could face bans in some locations. Europe and the UK have both launched investigations into the platform and warned of potential enforcement action.

How the Grok controversy unfolded

Earlier this week, Elon Musk’s AI company xAI came under regulatory scrutiny after reports showed Grok was being used to generate “sexualised images of children” as well as abusive images of women. Ofcom confirmed it had made direct contact with X and xAI, while an EU Commission spokesperson described the images as “unlawful”.

“This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe,” Thomas Regnier said to reporters on Monday.

An Ofcom spokesperson said the regulator had “made urgent contact with X and xAI to understand what steps they have taken to comply with their legal duties to protect users in the UK”.

Officials in India have also raised concerns with X over its AI image generation tools. Reporting from the BBC, Reuters and other outlets found that Grok was producing a range of images of women and children in inappropriately revealing clothing, using a feature referred to by the company as ‘spicy mode’.

While the mode does not allow full nudity, it does enable users to prompt for revealing clothing and sexualised poses. Researchers at French non-profit AI Forensics found that manipulated images included people under the age of 18, including some under the age of five.

Ofcom has since opened an investigation into Grok amid concerns that it was producing undressed images of real people. Although Ofcom contacted xAI on Monday (5 Jan), X had issued a warning a day earlier urging users not to use Grok to generate illegal material.

Musk also warned that anyone found to have used xAI to generate illegal content would “suffer the same consequences” as if they had created the images themselves.

The images generated by Grok appear to be in direct breach of X’s own policies, which state that the tool cannot be used to depict likenesses of people “in a pornographic manner”. Under the UK’s Online Safety Act, the creation or sharing of sexually explicit images of a person without their consent is illegal, and platforms are required to take “appropriate steps” to mitigate such content.

UK Technology Secretary Liz Kendall has called on Musk to take swift action against the proliferation of sexualised images created using Grok. Following contact from Ofcom, Kendall described the rise in use of Grok’s ‘spicy mode’ as “absolutely appalling”.

Reports have shown that the images, while not fully nude, depict real women and children in sexualised poses and skimpy clothing, with investigations finding that some images involved children as young as five.

“The UK government cannot and will not allow the proliferation of these degrading images,” Kendall said.

Recommended

• The Twitter Blue Tick is Back – With a Higher Price Tag for Some
• X Under Fire Over ‘Staggering Amplification of Hate’ During UK Riots
• Big Tech Faces EU Digital Services Act Reckoning

“Services and operators have a clear obligation to act appropriately. This is not about restricting freedom of speech but upholding the law.

“We have made intimate image abuse and cyberflashing priority offences under the Online Safety Act – including where images are AI-generated. This means platforms must prevent such content from appearing online and act swiftly to remove it if it does.”

She added that it was “absolutely right that Ofcom is looking into this as a matter of urgency” and said the regulator had her “full backing to take any enforcement action it deems necessary”.

In a statement, X said: “We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary.

“Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.”

Under the Online Safety Act, firms operating in the UK are required to take appropriate measures to prevent and address the spread of illegal content, including non-consensual sexual imagery, whether generated by humans or AI systems.