The social media network X has sparked widespread outrage after people used its AI chatbot Grok to change images of females by taking off their clothing.
Several examples have been seen where the technology has been used to remove clothing from women to make them appear in swimwear without consent, as well as to put them in inappropriate scenarios.
Company Reaction and Regulatory Stance
XAI, the organization that created Grok, offered no comment to inquiries, other than an auto-reply response stating "legacy media lies".
A government representative stated that legislation is being proposed to ban nudification software. Under a proposed criminal offence, anyone who supplied such tools would "receive a jail term and significant penalties".
"The platform has the ability to stop these types of abuse if they wanted to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Functionality
The regulator Ofcom said platforms must "evaluate the potential" of people accessing illegal content on their networks, but did not specify whether it was presently investigating X or Grok in regarding AI-generated pictures.
Grok is a no-cost AI assistant—with some paid-for features—which responds to X users' queries when they mention it in a tweet.
It is often used to provide reactions or more background to other users' remarks, but people on X are also can alter an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been condemned for enabling users to create pictures and footage with nudity and sexually suggestive content, and it was in the past accused of generating graphic content of public figures.
"X has been permitting the production and sharing of these pictures for months without taking any action and we have yet to see any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "portraying likenesses of people in a pornographic way".
In a comment, Ofcom said it was illegal to "create or circulate private pictures or child sexual abuse material" and clarified this included explicit AI images made with AI.
It said services such as X were mandated to take "suitable steps" to "minimize the danger" of users viewing illegal material on their platforms and to remove it quickly when they are notified of it.
