The social media network X has provoked widespread anger after people utilized its AI chatbot Grok to alter pictures of women by taking off their garments.
Numerous examples have surfaced where the technology has been used to remove clothing from women to make them appear in bikinis without consent, as well as to put them in sexually explicit situations.
Developer Response and Legal Stance
XAI, the firm that created Grok, did not respond to requests for comment, other than an auto-reply response stating "legacy media lies".
A Home Office official stated that legislation is being brought forward to outlaw nudification tools. Under a proposed law, anyone who distributed such tools would "face a prison sentence and substantial fines".
"The platform is capable of halting these forms of abuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said technology companies must "consider the danger" of people viewing illegal content on their platforms, but stopped short of confirming whether it was currently investigating X or Grok in regarding artificial intelligence pictures.
Grok is a no-cost chatbot—with some paid-for functions—which responds to X users' prompts when they reference it in a message.
It is often used to give reactions or additional background to other users' remarks, but people on X are also capable of alter an uploaded photo through its artificial intelligence picture modification function.
Background of Controversy and Policy
The tool has been condemned for enabling users to produce pictures and videos with nudity and sexually suggestive content, and it was earlier alleged to have been involved in creating graphic material of public figures.
"X has been enabling the generation and distribution of these images for months without taking any steps and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own terms of service forbids "depicting likenesses of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "generate or distribute deepfakes or illegal content" and stated this covered explicit AI images generated with AI.
It said services such as X were required to take "suitable actions" to "minimize the danger" of users coming across illegal material on their services and to delete it promptly when they learn of it.
