The online network X has sparked widespread backlash after individuals used its artificial intelligence assistant Grok to change images of women by stripping their garments.
Several cases have been seen where the technology has been employed to remove clothing from female subjects to render them in swimwear without permission, as well as to insert them in inappropriate scenarios.
Company Response and Regulatory Position
XAI, the organization that created Grok, did not respond to questions, other than an automatically-generated response stating "traditional news outlets lies".
A government official stated that legislation is being introduced to outlaw nudification technology. Under a upcoming law, anyone who provided such tools would "receive a jail term and substantial fines".
"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "operate with a lack of consequences."
Regulator Stance and Platform Capabilities
The regulator Ofcom said technology companies must "consider the risk" of people accessing unlawful material on their services, but did not confirm whether it was presently looking into X or Grok in relation to artificial intelligence pictures.
Grok is a no-cost chatbot—with some premium functions—which responds to X users' requests when they tag it in a message.
It is commonly used to provide commentary or further context to other users' remarks, but people on X are also capable of alter an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The system has been faulted for allowing users to produce images and videos with explicit content and sexually suggestive material, and it was earlier accused of making inappropriate material of celebrities.
"X has been enabling the generation and sharing of these images for a long time without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules forbids "showing likenesses of people in a graphic manner".
In a statement, Ofcom said it was unlawful to "generate or share deepfakes or child sexual abuse material" and confirmed this covered explicit AI images created with AI.
It said services such as X were required to take "necessary steps" to "reduce the risk" of users encountering unlawful material on their platforms and to take down it quickly when they are notified of it.