Grok can no longer undress women or generate child pornography, Dutch court says

The Court of Amsterdam has ruled that Grok is no longer allowed to generate or distribute nude images of women or pictures that show child sexual abuse material (CSAM).

If Grok keeps on producing nude photos or scantily clad women without explicit consent, X has to pay a penalty of €100,000 per day to advocacy group Offlimits, with a maximum of €10 million.

The creation and distribution of child pornography is prohibited as well, resulting in the same penalty.

Lastly, the Court of Amsterdam ruled that X is prohibited from offering the nudify functionality of Grok as part of the platform in the Netherlands.

The ruling of the Court of Amsterdam only applies in the Netherlands.

In February, two Dutch advocacy groups, Offlimits and Fonds Slachtofferhulp, filed a summary proceedings against X and Grok for undressing people at a user’s request.

The groups argued that images that were edited or created with AI tools are being used to bully, humiliate, or sexually harass victims online.

“Moreover, the accessibility of these types of tools makes it possible to create and distribute them on a large scale. What we are seeing is a slow-motion disaster that calls for action,” Offlimits Director Robbert Hoving said in a statement.

To prevent more women and children from becoming victims, waiting for X to solve the problem on its own wasn’t an option, the advocacy groups said.

Around the same time, the United Kingdom introduced an amendment to the Crime and Policing Bill, requiring tech companies to take action against the creation and distribution of non-consensual intimacy images within 48 hours.

“By requiring companies to remove non‑consensual intimate images within 48 hours, we are finally putting the onus where it belongs, on the tech firms with the power and resources to act. It’s a vital step towards making the online world safer, fairer, and more respectful for women and girls,” Liz Kendall, Secretary of State for Science, Innovation and Technology, said in a statement at the time.

Last week, the European Parliament greenlit changes to the AI Act banning so-called nudifier apps in the European Union.