The social media platform X has provoked major backlash after individuals utilized its AI chatbot Grok to alter pictures of women by taking off their garments.
Multiple examples have been seen where the technology has been employed to virtually strip women to show them in swimwear non-consensually, as well as to place them in sexualized situations.
Company Response and Legal Stance
XAI, the company that created Grok, offered no comment to requests for comment, aside from an auto-reply response stating "traditional news outlets lies".
A Home Office spokesperson stated that legislation is being proposed to ban nudification tools. Under a proposed criminal offence, anyone who distributed such tools would "be subject to a jail term and substantial fines".
"X or Grok could prevent these forms of abuse if they desired to," said a law professor, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The regulator Ofcom said technology companies must "evaluate the potential" of people viewing illegal content on their platforms, but did not specify whether it was currently examining X or Grok in regarding artificial intelligence images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' requests when they tag it in a post.
It is often used to provide reactions or more background to other users' remarks, but people on X are also capable of alter an uploaded photo through its AI image editing tool.
History of Controversy and Policy
The system has been faulted for allowing users to create photos and clips with nudity and sexualised material, and it was in the past accused of generating graphic material of public figures.
"The platform has been enabling the production and spread of these pictures for months without taking any action and we have yet to see any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of people in a pornographic fashion".
In a comment, Ofcom said it was unlawful to "make or circulate deepfakes or child sexual abuse material" and stated this covered sexual deepfakes generated with AI.
It said services such as X were obligated to take "necessary steps" to "lessen the risk" of users coming across unlawful material on their platforms and to take down it swiftly when they are notified of it.
