The online platform X has provoked widespread backlash after people employed its artificial intelligence assistant Grok to manipulate images of women by stripping their clothing.
Numerous examples have surfaced where the technology has been employed to virtually strip women to render them in swimwear without consent, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the firm behind Grok, remained silent to questions, except for an automatically-generated message stating "traditional news outlets lies".
A government official stated that new laws is being introduced to outlaw nudification technology. Under a new law, anyone who supplied such tools would "be subject to a prison sentence and significant fines".
"The platform is capable of halting these forms of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Capabilities
The regulator Ofcom said platforms must "consider the danger" of people accessing unlawful material on their platforms, but stopped short of confirming whether it was at this time examining X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost AI assistant—with some premium functions—which reacts to X users' requests when they tag it in a tweet.
It is commonly used to provide reactions or more context to other users' remarks, but people on X are also capable of alter an uploaded picture through its artificial intelligence picture modification function.
History of Criticism and Rules
The system has been criticised for enabling users to produce photos and videos with nudity and sexually suggestive content, and it was earlier alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been allowing the generation and sharing of these images for months without taking any steps and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own terms of service forbids "depicting likenesses of people in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "make or distribute private pictures or child sexual abuse material" and stated this included explicit AI images made with AI.
It said networks such as X were obligated to take "necessary measures" to "reduce the danger" of users coming across illegal content on their services and to delete it quickly when they learn of it.
