The social media platform X has caused major anger after people used its AI chatbot Grok to change photos of women by removing their clothing.
Several cases have emerged where the technology has been employed to remove clothing from female subjects to render them in bikinis without permission, as well as to insert them in inappropriate contexts.
Company Reaction and Regulatory Position
XAI, the company responsible for Grok, offered no comment to requests for comment, other than an standard response stating "legacy media lies".
A government official stated that new laws is being proposed to ban nudification software. Under a new law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Tool Capabilities
The communications watchdog Ofcom said tech firms must "evaluate the risk" of people accessing unlawful material on their platforms, but did not confirm whether it was presently looking into X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which reacts to X users' prompts when they mention it in a message.
It is frequently used to provide commentary or further context to other posters' comments, but people on X are also capable of modify an posted picture through its AI image editing feature.
History of Criticism and Policy
The tool has been condemned for enabling users to produce images and footage with nudity and sexually suggestive content, and it was in the past accused of creating inappropriate content of celebrities.
"X has been permitting the generation and spread of these images for months without taking any action and we have yet to see any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting likenesses of people in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or share non-consensual intimate images or child sexual abuse material" and clarified this covered sexual deepfakes made with AI.
It said networks such as X were mandated to take "appropriate steps" to "lessen the risk" of users coming across illegal material on their services and to delete it swiftly when they learn of it.
