There's no jailbreaking going on here. The filters are all functioning as intended when someone requests transparent, skimpy, impossibly thin, or skin-tone clothing even on posts that explicitly give context that a child is pictured there. It's on the service.
> Unlike other leading chatbots, Grok doesn’t impose many limits on users or block them from generating sexualized content of real people, including minors, said Brandie Nonnecke, senior director of policy at Americans for Responsible Innovation. Other generative AI technologies, including ones from Anthropic PBC, OpenAI and Alphabet Inc.’s Google, are “giving a good-faith effort to mitigate the creation of this content in the first place,” she said. “Obviously, xAI is different. It’s more of a free-for-all.”
If GIMP had AI features like this, I'd expect safeguards. It doesn't. All other AI tools have safeguards against this kind of bad behavior that are lacking in Grok.
As always with AI, the barrier to entry has evaporated. You can create nasty stuff with a pencil but you can't flood the internet mass producing nasty stuff with a pencil.
There are basic, obvious safeguards that are not in place here. That's why the software is to blame. If it was some sort of jailbreaking or circumvention, that'd be one thing. But given the owner himself is amplifying this, this borders on being an intended use case.
