The tool, which is called Kromix, bills itself as an “AI image styler.” A voice-over for one of the video advertisements, viewed by WIRED, promotes the tool as having “no restrictions,” saying “all the characters are real people” and describing it as “the AI that men actually use.” In one ad, a woman closely resembling a prominent female US politician is seen in front of a US flag and the flag of the president of the US above the caption “What if she moved?” She then winks, before the ad cuts to a scene of a woman with the same face performing in a pornographic video.
The Kromix app, which was hosted by Apple’s App Store prior to a request for comment from WIRED, invites paid users to upload photos so that it can use them in scenarios including “bedroom rape” and “Disney love.” Videos on the app also feature AI-generated pornographic videos of women wearing Spider-Man costumes. Requests for comment sent to a contact listed in the app went unanswered.
Pretty normal thing for Meta to do, isn’t it?
The Kromix app, which was hosted by Apple’s App Store prior to a request for comment from WIRED,
Apple has struggled to keep similar tools, which violate its policies against nonconsensual intimate imagery, nudifying, and sexually explicit content, off the App Store.
In January, as Grok was used to create around 3 million sexualized images of women and girls, officials in Sweden, England, and Northern Ireland were targeted by people using the technology, often using official images as a starting point.
Yes, this isn’t surprising for Meta to allow with no oversight, but it’s not just Meta. This shit is everywhere. And if we don’t keep calling it out, if we don’t keep pushing back and asking why it’s there, why it’s allowed, it will continue.
Yes, I agree. It’s great that people are still fighting this bullshit. I just stopped buying and using their products and services.
Same, I quit using Meta anything a while ago. There’s still so many people who use it!
I think it’s probably fine to hit anyone still using Facebook in the head with a brick.
Too bad we need those data centers to stay open, for all the valuable important work they do. Statisti Cally generated kiddie porn is just the cost of progress.
saying “all the characters are real people” and describing it as “the AI that men actually use.”
invites paid users to upload photos so that it can use them in scenarios including “bedroom rape” and “Disney love.”
Okay, seriously, what the fuck is this timeline? What is going on?
I’m scared. I need an adult.
I can’t help but think the near complete lack of moderation at meta is intentional considering they must be making ad revenue of this and other controversial aps.
Still, the bigger story is that this is happening at all, that it’s all women, and it’s all geared toward men. We can assume it’s selling if there’s this much of it, that it’s this hard to keep off platforms, and that’s fucking horrifying.
There’s plenty of moderation. It’s just not doing what you’d like it to.
Paul says that Meta has increasingly relied on AI to flag and vet advertisements, while laying off human moderators. “Ads are Meta’s primary product, and there appears to be zero quality control,” she says. “Meta is also making a lot of statements publicly, banking on their AI and the sophistication of it, while simultaneously saying they use AI to review their ad library and nothing’s perfect.”
I’ve had idiot friends who tried saying leftish things on there. There’s definitely still moderation.
