The social media network X has caused major backlash after individuals used its artificial intelligence assistant Grok to alter photos of females by stripping their clothing.
Multiple instances have been seen where the technology has been used to virtually strip women to show them in swimwear without consent, as well as to insert them in sexualized scenarios.
Company Response and Legal Position
XAI, the firm that created Grok, offered no comment to questions, except for an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being proposed to outlaw nudification technology. Under a proposed law, anyone who provided such tools would "receive a prison sentence and heavy penalties".
"The platform could prevent these types of misuse if they chose to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "evaluate the risk" of people seeing illegal content on their platforms, but stopped short of confirming whether it was presently investigating X or Grok in relation to AI-generated images.
Grok is a no-cost chatbot—with some premium functions—which answers X users' queries when they mention it in a message.
It is commonly used to offer reactions or additional context to other posters' remarks, but people on X are also able to alter an posted photo through its artificial intelligence picture modification feature.
Background of Criticism and Policy
The system has been faulted for permitting users to generate pictures and videos with explicit content and sexually suggestive content, and it was in the past accused of creating graphic material of public figures.
"X has been allowing the production and distribution of these images for a long time without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "portraying likenesses of persons in a pornographic way".
In a comment, Ofcom said it was unlawful to "create or circulate deepfakes or child sexual abuse material" and clarified this covered explicit AI images created with AI.
It said platforms such as X were obligated to take "suitable measures" to "minimize the risk" of users encountering illegal material on their services and to remove it promptly when they are notified of it.
