An amended class action lawsuit against X and xAI includes one of the most disturbing cases of AI-generated CSAM yet. A girl’s stepfather used Grok to create 7,000 sexually explicit images from a single photo taken when she was 11 years old. Then he killed himself after police found the images.
Grok didn’t flag any of it. The system let him generate extreme images depicting incest and rape without intervention. xAI’s safety system only kicked in when he typed a “gang rape” prompt. That triggered a CyberTip to NCMEC, which alerted law enforcement.
But even then, the lawsuit alleges xAI obstructed the investigation. Despite mandatory reporting requirements, xAI refused to share IP address or user information for weeks. The complaint says xAI “obstructed this investigation at every turn.”
The stepfather was eventually arrested after cops got a warrant. A forensic review found the 7,000 images. Two days after he was released on bail, he shot himself. The girl — identified as Jane Doe 4 — now struggles with anxiety, depression, and suicidal thoughts.
The lawsuit says this isn’t an outlier. NCMEC found that 90% of xAI’s CyberTipline reports weren’t actionable because xAI refused to include user information. The complaint also adds Stability AI as a defendant, alleging their open-weight models were trained on CSAM and serve as the basis for “nudify” apps.
Grok’s lax safeguards — which let users make “spicy” requests to undress images — have harmed tens of thousands of kids, according to researchers. xAI’s only action so far has been to charge for the feature.
