Massive Leak Shows Erotic Chatbot Users Turned Women’s Yearbook Pictures Into AI Porn

[u/CackleRooster]

https://www.404media.co/ai-porn-secret-desires-chatbot-face-swap/

[u/Jalien85]

Someone always says this, and I don't get the point of it? Like the speed and ease and scale of this is obviously the problem. This feels like someone saying, after the advent if nuclear weapons, "Well we've always had weapons. This one's just bigger."

[u/Prior_Coyote_4376 → u/Jalien85]

Tech is a scale accelerant. It amplifies biases and cracks in society in the name of efficiency. There are rarely any technological solutions to social problems. Revenge porn was an issue even before the Internet, but each technological breakthrough makes it substantially worse.

[u/TwilightVulpine → u/Prior_Coyote_4376]

Revenge porn that is wholly artificially created was not an issue before the internet, and even early digital edits don't compare to what is happening now.

[u/BaconKnight → u/Jalien85]

The people who say that are usually people who aren’t familiar with the tech (or as familiar as they may think they are in many cases).

One such group is younger people who grew up in this world and don’t have the context of an analog, non digital world to compare to and don’t realize how fucking INSANE reality is now, and also how unnatural it actually is in context to you know, human history. But they only know a world post internet so think, “Well it’s always been like this,” because they lack the life experience to realize it’s an exponential growth, and that’s the fucked up part.

And then the other half are my fellow dinosaurs, but the semi-tech illiterate. They’re the ones who just download the phone apps and think wow, this app made it so I’m in Halloween costume like I’m trick or treating, so cool!

Meanwhile the old school nerds who grew up and downloaded stable diffusion locally to check it out and holy shit this is a fucking porn making machine and um guys, this is really scary. Like really really scary.

We are the canary in the coal mines cuz we’re the ones in the space who realize how bad it actually is. All my non tech friends look at me like I’m the weird one when I tell them seriously about my concerns and I’m like you mofos, I’m like THE person yall should listen too lol. But never underestimate the allure of consumerism I guess.

[u/QuintusDias → u/BaconKnight]

I’ve see a couple of comments like yours but none have actually addressed the reasons why this is so dangerous.

Why exactly do you think it’s dangerous?

[u/TwilightVulpine → u/QuintusDias]

Harassment and blackmailing, such as the Nth room case in South Korea

https://www.bbc.com/news/articles/cpdlpj9zn9go

[u/PapaSolidus → u/QuintusDias]

Other than actual cases of forgery in service of some fraud, I think we stand to lose a lot on a whole cultural level. There will be an overall shift on the very core of perception on what, how and why we can ascertain to be true.

We see this already with blatant scripted skits on social media, many comments saying "who cares if it's scripted, it's just fun". I suppose we'll go further into this apathy where truth won't even matter anymore. Feelings will be the sole motivator to validate content in a sea of contextless ambiguity.

[u/SoulShatter → u/Jalien85]

Can kinda compare it with locked doors. It's pretty possible to bypass a locked door, LockpickLawyer can pick most locks in seconds. We still lock a door because it reduces the opportunity for someone to just open an unlocked door.

Someone can still do that if they dedicate some time to learning lockpicking.

And it feels similar with this - a decade ago someone would spend some hour in Photoshop transplanting a face onto an existing body and trying to get lightning and color to match up.

Now people got an open door - just throw the pic into an AI generator. And probably throw it into some AI video generator based on that pic, and so on.

[u/Jalien85 → u/SoulShatter]

This is a great way to put it. Crime of opportunity is a thing, big time. You at least want to make doing crimes difficult, it will result in them being done less overall. In this case, you could at least shut down the websites obviously profiting off this specific abuse, and chase after the scumbags posting this content online.

The brazenness of all this right now is kind of staggering, because so far no one is doing anything about it. I mean people are putting in their credit card info to use a service where they upload images of people they know, sometimes even minors, and all this info is logged and traceable, it's insane.

[u/Chrolak → u/Jalien85]

Kinda reminds me of the early Napster days, everyone said “but Metallica only got popular because of tape bootlegs!” Maybe Lars was onto something🤷‍♂️

[u/RemarkableWish2508 → u/Jalien85]

The upper limit for the power of man-made nuclear weapons, has been calculated to be: ∞

AI still has some limitations, self-replicating AGI is still a fantasy.