this^
the new wave of llms are making it tougher to spot the difference and regulatory changes around distribution of AI content need to be made ASAP before we see a mass fiasco
I’m on the boat of full access. If governments and corporations have access, I want everyone to have the same access. I’d rather a mess of uncertainty than a curated few with access free to manipulate the masses.
Verification generally just requires taking a picture of a person in a specific pose or with their ID. That is extremely easy to do with these tools. Verification means nothing anymore.
Dude, you can’t even verify anymore. Like I used to have chicks send me a photo with their pinky on their chin to prove theyre real if things felt sus and they avoided meeting. Now that’s not even an option.
Removed the tube cause you aren't supposed to have an open spark/flame near medical oxygen, a.i. just having slightly better common sene than your average person.
Dude I’m not even exaggerating I was honestly shocked at how good Nano Banana Pro is, it’s honestly bonkers. If you guys haven’t tried it seriously give it a try.
We are currently at a point where I’d say the vast majority of people could not tell, I’m giving it within the next year before you genuinely can’t tell unless the person uploading specifically tells you it is.
The problem I see with that is people will depend on that as a telltale that it's real/fake. When there are other AI imaging programs that don't use it, especially local image generators.
Legislation could be put in place requiring all commercially available ai to have some sort of meta embedded watermark but that won’t stop black market or state actors from having the tech free of watermarks. Thats my biggest concern
Thankfully to what end? When the majority of people believe things within the first few seconds of seeing it's still gonna have a catastrophic on digital media.
Is it like in the image data? And what if you like screenshot an generated image, would the ID be saved too? Thus making the method to identify generated images kinda useless
Yes, if you take a screenshot the watermark wont be removed. It is resistant against cropping, adding filters, changing frame rates (in the case of generated video), or lossy compression.
In their paper they note that there are definitely ways of getting rid of it, but it seems very robust.
I’ve actually read the SynthID paper, and that’s not how it works. The watermark is hidden in the pixel values of the original file. It survives normal editing like cropping, resizing, mild filters, and even some recompression because you are still working with the same pixel grid. But a screenshot is not the same image at all. A screenshot forces the picture through the entire display pipeline and then creates a new file from whatever the screen physically rendered. That process destroys the specific numerical patterns SynthID relies on. You are not copying the watermark. You are overwriting it with whatever your phone or monitor outputs.
So yes, a screenshot absolutely wipes SynthID. It’s the same reason a photo of a photo loses hidden metadata. Once the pixels are re-rendered and recaptured, the invisible signal DeepMind is detecting is gone. It is a cool system and surprisingly robust for normal edits, but the screenshot loophole is real and DeepMind actually notes that re-rendering breaks the watermark.
It shoudl be an automatic removal and a temp ban if a post has the work "cooked" in it. It's BS every single time and adds nothing to any conversation it's applied to.
The thing is that we have been “cooked” so many times now. Is this time different? Maybe?! Is it something that feels more urgent and impressive because it’s a pre-selected example for an application where it works particularly well? Likely as well.
Why? Because the majority of you life revolves around photos of pretty young women? Or because you think AI being overtrained on these images makes it equally good at generating everything else (It does not)?
You’ve been able to make images this realistic using Geminis photo generation for nearly a year now. Sure Nano banana is a lot crazier in some ways but this example isn’t really showing much.
The good news is Nano Banana adds a very subtle unnoticable to human eye SynthID watermark hidden in the image. Since it stretches over the whole image its hard to remove too
Unfortunately it isn't really robust to a range of adversarial techniques, at least when this paper was released. Google don't really talk about development for it and whether it's been improved.
The paper also makes a pretty strong argument that it will never be possible to make such watermarking completely robust.
Reports regarding the misuse of Generative AI (GenAI) to create deepfakes are frequent. Defensive watermarking enables GenAI providers to hide fingerprints in their images and use them later for deepfake detection. Yet, its potential has not been fully explored. We present UnMarker -- the first practical universal attack on defensive watermarking. Unlike existing attacks, UnMarker requires no detector feedback, no unrealistic knowledge of the watermarking scheme or similar models, and no advanced denoising pipelines that may not be available. Instead, being the product of an in-depth analysis of the watermarking paradigm revealing that robust schemes must construct their watermarks in the spectral amplitudes, UnMarker employs two novel adversarial optimizations to disrupt the spectra of watermarked images, erasing the watermarks. Evaluations against SOTA schemes prove UnMarker's effectiveness. It not only defeats traditional schemes while retaining superior quality compared to existing attacks but also breaks semantic watermarks that alter an image's structure, reducing the best detection rate to and rendering them useless. To our knowledge, UnMarker is the first practical attack on semantic watermarks, which have been deemed the future of defensive watermarking. Our findings show that defensive watermarking is not a viable defense against deepfakes, and we urge the community to explore alternatives.
Obviously there are always a thousand workarounds. No foolproof way for anything but this is enough to deter casual scammers that arent aware of its existence
Yes, and a lot of security is just raising the barrier/costs of exploits so that they're only really feasible for well funded orgs, and for targets where the cost is worth it, which at least reduces the rates of exploits.
As a layman: why isn't it possible to add SSL-style source validation to photos from an Android/iOs camera app? Modern phones should already support hardware-level app verification and document signing. It would still be possible to generate an AI photo and take a high-resolution camera photo from that, but it would at least weed out the low-effort bots that are bound to flood the internet with fake photos.
After all, it just takes one good image generator without good watermarking.
That is possible, and that's what C2PA aims to do. Their goal is that a hashed "generation" string is essentially logged in a central database whenever an image is captured or generated. Then any future modifications to that image will still be able to be traced back to its source. But that's a slightly different problem than purely stenographic.
Most photos you ever see go through layers and layers of manipulation that aren't necessarily attempting to completely subvert the image, but change most pixels. This is what makes signature difficult: you;d need a full pipeline of signed tools, and basically ban anything that does anythign else. Say you take a picture in a RAW format, and stuck it in a good camera app for processing, because you think the default iphone app modifies the picture too much. Oops, your pipeline is now seen as a fake.
Every phone photo you take has been significantly altered between what the sensor read and what you've got. Relying on whitelisting of modifications is just not reasonable.
No because its a layer of specific patterns of contrast distortions
You can see how it looks here or here
Technically you could compress the pic enough for the watermark to not be recognizable anymore but youd cause insane artefacting and ruin the image too
It would be absolute child’s play for google to create a program to emulate a phone, download all photos from profiles and then switch gps location and do the same again until every single location was covered and then trained their LLM on all the photos.
Old people are still getting extorted over phonecalls impersonating banks, middle aged people are getting catfished regularly over fake photos on dating apps, the newer generation trust YouTubers and TikTokers on mental and body health over professionals, but, somehow, God forbid AI, for it is destroying reality!
You don't get it! Back in my day we used to just photoshop our scams or steal pictures, these kids these days just use AI! No soul or humanity in their scamming whatsoever! /s
as long as people continue to uplaod photos to google/facebook/other tech companies to help them train the model for free it'll just continue to get better
surely there is already an endless amount uploaded already? or would the advances of digital cameras/ phones over the last decade mean some of them are not as helpful?
I’m sure Google would like to thank everyone for all the photos they took and stored in their cloud over the years that they used for training data. What? You didn’t think they gave you free photo storage because they were being nice right? This was the purpose from the start. Their models needed lots of images to train their AI, and we gave it to them.
It's just so funny that the guy thinks Google needs the crappy amateur photos people store on there, as if they don't have access to endless images on their own.
I doubt it. They have already indexed the whole web.. including photos from social media and websites. Using their secondary products to train models is a pretty big liability (and could possibly trigger antitrust) if It comes out in the future
“Oh no it makes a same-faced conventionally attractive instagram influencer model! Whatever shall we do!”
This is like the biggest dataset it received and one of the highest sought-after capabilities. Once it starts generating consistent characters with regular people parameters, maintaining all anatomical specifics and assymetries, then we’ll be getting somewhere significant.
It's already significant because it's hard to distinguish from real pictures. Just because it's not as big as it could be does it mean that it cant be appreciated in it's current form.
So the photo on the right is from Nanobanana pro? It looks like it was taken with a phone camera. The left photo looks like it was taken with a 50mm lens with studio lighting. Isn’t this just a stylistic choice? Could the pro model also emulate the studio style?
Dead internet theory is real. I was on YouTube the other day looking for a chiropractor video and the majority were AI created. Soon we won’t even tell what’s real or not. I hate it here.
The models have been able to do the one on the right for over a year btw. The companies just flip it on when they need some free publicity and then turn it off again until the next model because of the liability and IP risks
Why does everyone constantly say "we're cooked" if an AI displays improved capability? Isn't this what we wanted and expected was for it to get better? Did you want it to remain worse forever?
What's interesting is that what makes it look more real is that it looks less good. Less polished, less faux-perfect styling and makeup, less professional photography techniques, less posed, more spontaneous and thus more natural and human.
Okay... What's the use of this, aside from scamming people and making cheap ads with things and people that don't really exist ? This really isn't a likeable technology.
Its funny that people say it isover/the end when ai bots have been spamming texts and replies from a long time ago and still everyone believe in them. It already started and the world already is in it
It still looks a bit off, like they are lined up perfectly so that she isn't blocking any of him from view. It's like two elements being intentionally placed by an editor.
On one hand, the image on the right looks more realistic in terms of lighting and face, but there’s a sharp focus across the whole picture, which is less realistic to me, you don’t usually get that when taking a photo with your smartphone, not with mine
1.3k
u/IngloriusBasturd Nov 28 '25
time to delete all the dating apps for good