The EU Commission is opening an investigation to see if X’s Grok AI chatbot violated the Digital Services Act over the dissemination of manipulated sexualise images over the social media platform.

The formal investigation follows the torrent of sexualised deepfakes generated by Grok prompted by users under its ‘spicy mode’, which enabled imges of people to be ‘undressed’ and clad in revealing clothing or in sexualised positions.  The mode was used on images of real people, mainly women, and included images of children.

The investigation seeks to clarify is X acted appropriately under EU law to mitigate the risks associated with Grok and its function

A senior EU official told reporters on Monday that the Commission so far has reasons to believe that X may not have carried out an ad hoc assessment of Grok when it rolled out its spicy mode, and the changes it made, though welcomed, wee not enough to resolve the issues and their associated risks.

The EU Commission already called the sexualised images of women and children generated by Grok “appalling”, now the Commission will see if X will pay a financial price for the pictures.

“Non-consensual sexual deepfakes of women and children ‌are a violent, unacceptable form of degradation,” EU technology chief Henna Virkkunen said in a statement about the investigation

Recommended

• X Stops Grok Generating Revealing AI Deepfakes Following Backlash
• X Under Fire Over ‘Staggering Amplification of Hate’ During UK Riots
• X Limits Grok AI Images to Subscribers Following Deepfake Outcry

X has restricted Grok’s spicy mode abilities based on user location, and has said that generated images of ‘undressed’ people will be banned in regions where it is illegal. Still, it has not revealed what areas would see this restriction, or what laws would deem the tech illegal.

“With this investigation, we will determine whether X has met its legal obligations under the DSA, or whether it treated rights of European citizens – including those of women and children – as collateral damage of its service,” Virkkunen said.