The online platform X has provoked major outrage after people utilized its AI chatbot Grok to alter images of females by taking off their attire.
Multiple instances have emerged where the technology has been used to digitally undress women to render them in bikinis non-consensually, as well as to put them in inappropriate contexts.
Company Response and Regulatory Stance
XAI, the company that created Grok, offered no comment to requests for comment, other than an automatically-generated message stating "traditional news outlets lies".
A government representative stated that new laws is being introduced to outlaw nudification technology. Under a new criminal offence, anyone who provided such tools would "be subject to a prison sentence and substantial fines".
"X or Grok could prevent these forms of abuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "evaluate the potential" of people viewing unlawful material on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some premium features—which reacts to X users' prompts when they mention it in a post.
It is often used to offer reactions or additional context to other users' remarks, but people on X are also capable of alter an posted photo through its artificial intelligence picture modification tool.
History of Controversy and Policy
The chatbot has been condemned for enabling users to produce pictures and clips with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in generating sexually explicit content of celebrities.
"The platform has been enabling the creation and spread of these images for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting images of persons in a graphic fashion".
In a statement, Ofcom said it was unlawful to "make or distribute private pictures or illegal content" and clarified this covered explicit AI images generated with AI.
It said networks such as X were obligated to take "appropriate measures" to "minimize the danger" of users coming across unlawful content on their platforms and to remove it promptly when they are notified of it.
