The online network X has provoked widespread backlash after people employed its artificial intelligence assistant Grok to manipulate pictures of women by removing their attire.
Several examples have emerged where the system has been used to virtually strip female subjects to make them appear in swimwear without permission, as well as to place them in inappropriate contexts.
Company Response and Legal Position
XAI, the firm responsible for Grok, offered no comment to inquiries, aside from an auto-reply message stating "legacy media lies".
A government representative stated that legislation is being introduced to ban nudification software. Under a proposed law, anyone who distributed such technology would "face a prison sentence and significant fines".
"X or Grok has the ability to stop these forms of misuse if they chose to," said Clare McGlynn, adding they "seem to have impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said platforms must "assess the risk" of people seeing unlawful material on their networks, but did not specify whether it was currently looking into X or Grok in regarding artificial intelligence images.
Grok is a free chatbot—with some additional features—which responds to X users' queries when they mention it in a message.
It is frequently used to offer reactions or further context to other posters' comments, but people on X are also able to modify an uploaded photo through its artificial intelligence picture modification tool.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to generate photos and videos with nudity and sexually suggestive material, and it was earlier accused of generating graphic material of public figures.
"The platform has been permitting the generation and distribution of these images for a long time without taking any measures and we have not witnessed any intervention by oversight bodies," the expert added.
XAI's own acceptable use policy prohibits "depicting likenesses of individuals in a pornographic manner".
In a comment, Ofcom said it was against the law to "make or distribute non-consensual intimate images or child sexual abuse material" and clarified this included explicit AI images created with AI.
It said services such as X were obligated to take "necessary measures" to "minimize the risk" of users encountering unlawful material on their platforms and to delete it quickly when they learn of it.
