Dutch Court Bans AI Model for Generating Nonconsensual Nude Images

Amsterdam District Court Orders xAI to Comply with Dutch Regulations

A Dutch court has issued an order requiring Elon Musk’s xAI to stop generating and distributing nude images of people without their consent in the Netherlands. The Amsterdam District Court ruled that xAI’s Grok artificial intelligence tool and the X platform hosting it are prohibited from creating and distributing “sexual imagery” featuring individuals who are “partially or wholly stripped naked” without explicit permission.

The decision was made in a civil suit filed by Offlimits, a Dutch centre monitoring online violence, in cooperation with the non-profit Victims Support Fund. The lawsuit centered on Grok’s feature allowing users to create hyper-realistic deepfake montages of naked women and children using real photos.

xAI lawyers had argued that it was impossible to guarantee that abuse on its platform could be prevented, citing measures taken by the company in January to restrict image editing features for paid subscribers. However, the court found reasonable doubt over the effectiveness of these measures.

According to the court website, a video produced by Grok shortly before a hearing demonstrated the limitations of xAI’s current safeguards. Offlimits director Robbert Hoving stated that the burden lies with the company to ensure its tools are not used to create and distribute non-consensual sexual images, including those involving children.

The court has imposed fines of 100,000 euros ($115,350) per day for non-compliance. The European Parliament had previously approved a ban on artificial intelligence systems generating sexualised deepfakes, following global outrage over non-consensual Grok-produced nudes.

Source: Al Jazeera