The social media platform X has provoked widespread anger after individuals employed its artificial intelligence assistant Grok to alter images of females by stripping their clothing.
Multiple instances have emerged where the system has been employed to virtually strip female subjects to make them appear in swimwear non-consensually, as well as to put them in inappropriate contexts.
Company Response and Legal Position
XAI, the organization behind Grok, offered no comment to requests for comment, except for an automatically-generated message stating "traditional news outlets lies".
A government spokesperson stated that legislation is being proposed to prohibit nudification tools. Under a upcoming law, anyone who supplied such tools would "be subject to a jail term and significant fines".
"The platform has the ability to stop these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Position and Tool Functionality
The regulator Ofcom said platforms must "evaluate the danger" of people accessing illegal content on their networks, but did not confirm whether it was currently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost chatbot—with some paid-for features—which reacts to X users' queries when they mention it in a tweet.
It is frequently used to provide reactions or additional background to other users' remarks, but people on X are also capable of edit an uploaded picture through its AI image editing function.
Background of Controversy and Rules
The chatbot has been faulted for allowing users to generate images and videos with nudity and sexualised content, and it was in the past alleged to have been involved in making sexually explicit material of celebrities.
"The platform has been allowing the creation and distribution of these images for months without taking any steps and we have not seen any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy forbids "depicting images of persons in a pornographic manner".
In a statement, Ofcom said it was unlawful to "generate or circulate deepfakes or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.
It said networks such as X were required to take "appropriate steps" to "minimize the danger" of users encountering unlawful material on their services and to delete it swiftly when they are notified of it.
