Elon Musk's social media network X has provoked major outrage after users employed its artificial intelligence assistant Grok to change pictures of women by taking off their attire.
Several cases have been seen where the technology has been used to digitally undress female subjects to show them in swimwear without consent, as well as to put them in sexualized situations.
Company Response and Regulatory Stance
XAI, the firm that created Grok, offered no comment to questions, other than an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to outlaw image stripping tools. Under a new law, anyone who provided such tools would "be subject to a prison sentence and substantial fines".
"The platform could prevent these types of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Stance and Tool Capabilities
The regulator Ofcom said tech firms must "consider the risk" of people seeing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in relation to AI-generated pictures.
Grok is a no-cost artificial intelligence—with some additional functions—which answers X users' queries when they mention it in a message.
It is commonly used to provide commentary or further background to other users' comments, but people on X are also capable of alter an uploaded photo through its AI image editing function.
History of Criticism and Rules
The tool has been criticised for permitting users to produce images and clips with nudity and sexually suggestive content, and it was in the past alleged to have been involved in creating inappropriate content of public figures.
"The platform has been permitting the generation and sharing of these images for months without taking any steps and we have not seen any action by regulators," the expert added.
XAI's own usage rules forbids "portraying likenesses of people in a graphic manner".
In a comment, Ofcom said it was against the law to "generate or share private pictures or illegal content" and confirmed this applied to sexual deepfakes generated with AI.
It said platforms such as X were mandated to take "suitable actions" to "reduce the danger" of users encountering unlawful material on their services and to take down it swiftly when they are notified of it.
