The online platform X has provoked significant anger after users used its AI chatbot Grok to alter images of women by taking off their garments.
Numerous instances have been seen where the tool has been employed to virtually strip female subjects to render them in swimwear non-consensually, as well as to put them in inappropriate situations.
Company Reaction and Regulatory Stance
XAI, the company behind Grok, offered no comment to questions, aside from an standard message stating "legacy media lies".
A government representative stated that new laws is being proposed to prohibit image stripping software. Under a proposed law, anyone who provided such technology would "be subject to a prison sentence and heavy penalties".
"The platform could prevent these forms of abuse if they chose to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "assess the potential" of people accessing unlawful material on their networks, but stopped short of confirming whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a free artificial intelligence—with some premium functions—which reacts to X users' prompts when they mention it in a tweet.
It is often used to offer reactions or further background to other posters' comments, but people on X are also capable of modify an posted photo through its AI image editing feature.
History of Controversy and Rules
The system has been condemned for permitting users to produce photos and videos with nudity and sexually suggestive material, and it was previously alleged to have been involved in creating graphic content of celebrities.
"X has been permitting the generation and distribution of these images for months without taking any measures and we have not witnessed any action by regulators," the expert added.
XAI's own usage rules forbids "portraying images of individuals in a pornographic fashion".
In a comment, Ofcom said it was illegal to "make or distribute deepfakes or illegal content" and clarified this included explicit AI images made with AI.
It said services such as X were obligated to take "necessary actions" to "minimize the risk" of users encountering illegal material on their platforms and to delete it quickly when they become aware of it.
