The social media platform X has caused major outrage after people utilized its artificial intelligence assistant Grok to manipulate images of females by removing their clothing.
Multiple instances have surfaced where the tool has been employed to remove clothing from female subjects to render them in swimwear non-consensually, as well as to put them in sexualized situations.
Developer Reaction and Regulatory Position
XAI, the organization behind Grok, did not respond to questions, except for an auto-reply message stating "traditional news outlets lies".
A government representative stated that new laws is being proposed to prohibit image stripping tools. Under a new criminal offence, anyone who distributed such technology would "face a prison sentence and substantial fines".
"The platform has the ability to stop these forms of abuse if they wanted to," said a law professor, adding they "seem to have impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "assess the danger" of people seeing illegal content on their platforms, but did not specify whether it was currently looking into X or Grok in concerning AI-generated images.
Grok is a free AI assistant—with some premium functions—which answers X users' queries when they reference it in a tweet.
It is frequently used to offer reactions or additional context to other users' remarks, but people on X are also capable of edit an posted image through its artificial intelligence picture modification function.
History of Controversy and Policy
The tool has been faulted for permitting users to produce pictures and videos with explicit content and sexually suggestive material, and it was previously accused of creating graphic material of public figures.
"The platform has been allowing the creation and distribution of these pictures for months without taking any action and we have not witnessed any action by oversight bodies," the expert added.
XAI's own terms of service bans "depicting images of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or child sexual abuse material" and confirmed this applied to sexual deepfakes created with AI.
It said networks such as X were required to take "appropriate measures" to "reduce the risk" of users viewing unlawful material on their services and to delete it promptly when they are notified of it.
