xAI, the parent company of AI chatbot Grok, is being sued by three teenagers in the US for allegedly generating sexually explicit deepfake images of them without their consent or knowledge, breaking child abuse and child pornography laws.

The teenagers, based in Tennessee and suing in California, allege that Elon Musk’s xAI did not take steps to prevent Grok from generating child sex abuse material, “shattering” their lives as a result.

“Nearly all the companies creating, marketing, and selling AI recognized the dangers of such a tool and chose to enact industry-standard guardrails that would prevent the use of their products child sex predators. xAI did not,” the suit complaint states.

“Instead, xAI — and its founder Elon Musk — saw a business opportunity: an opportunity to profit off the sexual predation of real people, including children.”

xAI came under international fire for Grok’s ‘spicy mode’ which allowed users to generate illicit images of people without their knowledge or consent, requesting for images to be ‘undressed’ or ‘scantily clad.’ Women were targeted without their consent or knowledge, as were children, sparking outrage.

Instead of outright disabling ‘spicy mode’, Musk’s company merely prohibited it in countries where it was explicitly outlawed – forcing nations to race to make deepfake images formally illegal.

The current lawsuit alleged that xAI knowingly broke child pornography laws, claiming the firm was aware it enabled the creation, possession, and distribution of CSAM on its systems and servers. The suit aims to gain class action status, which means thousands could claim on the case.

The case centres around incidents that took place late last year, when one teenage plaintiff – Jane Doe 1 – was informed that nude images of her were circulating on a Discord server.

Police were able to arrest the creator and distributor of these images, finding that they processed images of the two other plaintiffs, as well as around 15 other teenage girls, many of whom went to the same high school. The images were sexualised and in some cases showed the girls fully nude.

The perpetrator used the images to trade for other child sexual abuse material on Telegram.

Recommended

• X Stops Grok Generating Revealing AI Deepfakes Following Backlash
• EU Opens Investigation Into X Over Grok AI Deepfakes
• X Under Fire Over ‘Staggering Amplification of Hate’ During UK Riots

The lawsuit claims that the images were generated using a third-party app that licenses Grok’s image-generation technology under a different brand by paying xAI. While it appears the lawsuit does not accuse the perpetrator of using Grok directly, the availability of the technology for such use is where liability hinges.

The case claims that xAI failed in several ways to block its technology from being used to generate CSAM, including safeguards to reject user prompts for sexual material, blocking any accidental material, checking databases for CSAM, and ensuring any harmful material is taken down promptly for victims.

The lawsuit points to xAI’s promotion of Grok’s ‘spicy mode’, and calls out its lack of safeguards.

While Grok does say that users should avoid “creating or distributing child sexual abuse material” in its system prompt instructions, the lawsuit claims this is easy to bypass and not enough to prevent harms.