Are we really surprised that the same tech promising utopia is also being weaponized for the oldest kinds of human depravity?
Generative AI, bless its silicon heart, has found a new calling: sexualized violence. Specifically against women and girls. Men, with their boundless creativity—or lack thereof—are conjuring up non-consensual deepfakes. They’re undressing digital avatars. Then they’re slinging these atrocities around social media like confetti. And let’s not forget the AI chatbots, spewing sexist bile on command. This isn’t new. It’s just patriarchy with a better GPU. The digital sphere mirrors the real one. It’s just faster, dirtier, and infinitely more scalable.
And the fallout? Devastating. Victims endure psychological and physical torment. Their recourse? Pathetic. These deepfakes aren’t just embarrassing; they’re tools of intimidation. Designed to silence women, to push them out of public life, out of politics. A digital muzzle.
Right now, the regulatory landscape is a mess. A patchwork quilt of EU and national rules, all loosely stitched together, with gaping holes. Germany, for instance, doesn’t even have reliable data on digital gender-based violence. Or a proper legal framework. So, the wolves are not only in the henhouse; they’re designing the henhouse.
The EU’s AI Act is supposedly being overhauled. The AI Omnibus procedure. A chance to fix things. But guess what? Some are trying to gut the safety features. Meanwhile, a ban on AI systems that churn out non-consensual sexualized deepfakes? That’s a rare glimmer of hope. A proposal that actually makes sense. It would finally hold AI companies responsible. A long overdue step.
The Deepfake Ban: A Balancing Act
But a ban isn’t a magic wand. It needs nuance. Consent needs a precise definition. Liability must be crystal clear. And safeguards can’t cripple legitimate AI development or open-source innovation. We can’t throw the baby out with the bathwater.
Not every AI image generator is a weapon. The focus should be on tools that explicitly facilitate abuse. Those lacking safety measures. Those that offer no escape from misuse. Even systems with safeguards aren’t foolproof. Clever folks will always find ways around them. When they do, they should be held liable. The position paper from AlgorithmWatch spells it out: consent needs to be free, informed, context-specific, and explicit. No applying it to non-real people. Liability should only fall on systems without adequate safeguards. And don’t mess with open-source AI. It’s not the enemy here.
We have summarized our detailed perspective on the matter in a position paper: Precisely define consent: free, informed, context-specific, and explicit; no application in the context of content of non-real people. Clearly assign liability: Only collect data from AI systems that do not have adequate safeguards. Avoid unintended side effects: Do not jeopardize open-source AI development through overregulation. Strengthen security measures: Implement adequate safeguards with continuous monitoring and reporting mechanisms. Consent requests: Make them mandatory for depictions of real people.
Platforms: Still Dodging Responsibility?
And the platforms? Oh, they’re still playing catch-up. Or perhaps, more accurately, they’re still amplifying the problem. Their algorithms are designed for engagement, for emotional triggers. Deepfakes thrive on that. They spread like wildfire. Platforms also become conduits for problematic apps, advertising them, even sharing tips on how to turn innocent LLMs into ‘non-consensual sexualization tools’ (NSTs). It’s a free-for-all.
This isn’t just about banning a few specific tools. It’s about a fundamental shift in accountability. AI companies can’t just shrug their shoulders. Platforms can’t claim ignorance. The perpetrators, armed with these tools, must face consequences. Without strong enforcement, any ban is just paper-thin. A gesture. And women fighting digital sexual violence deserve more than gestures.
There’s a disturbing historical echo here. Every technological leap—the printing press, photography, the internet—has been met with new forms of exploitation. And each time, the law and societal norms lag behind. We’re playing whack-a-mole with digital depravity. The current push for an AI Act ban on deepfake creation tools is a necessary, albeit belated, step. But it’s far from the finish line. The fight for digital safety is a marathon, not a sprint. And frankly, we’re still stuck at the starting gun.
🧬 Related Insights
- Read more: What is AI Copyright Law?
- Read more: Biglaw Layoffs: Not 2009, But Still Wary
