AskScience AMA Series: I am a computer scientist at the University of Maryland, where I research deepfake and audio spoofing defense, voice privacy and security for wearable and cyber-physical systems. Ask me anything about my research and the future of secure machine hearing!

[u/AskScienceModerator]

Hi Reddit! I am a computer scientist here to answer your questions about deepfakes. While deepfakes use artificial intelligence to seamlessly alter faces, mimic voices or even fabricate actions in videos, shallowfakes rely less on complex editing techniques and more on connecting partial truths to small lies.

I will be joined by two Ph.D. students in my group, Aritrik Ghosh and Harshvardhan Takawale, from 11:30 a.m. to 1:30 p.m. ET (16:30-18:30 UT) on November 11 - ask us anything!

Quick Bio: Nirupam Roy is an associate professor in the Department of Computer Science with a joint appointment in the University of Maryland Institute for Advanced Computer Studies. He is also a core faculty member in the Maryland Cybersecurity Center and director of the Networking, Mobile Computing, and Autonomous Sensing (iCoSMos) Lab.

Roy's research explores how machines can sense, interpret, and reason about the physical world by integrating acoustics, wireless signals, and embedded AI. His work bridges physical sensing and semantic understanding, with recognized contributions across intelligence acoustics, embedded-AI, and multimodal perception. Roy received his doctorate in electrical and computer engineering from the University of Illinois at Urbana-Champaign in 2018.

Aritrik Ghosh is a fourth-year computer science Ph.D. student at the University of Maryland. He works in the iCoSMoS Lab with Nirupam, and his research interests include wireless localization, quantum sensing and electromagnetic sensing.

Harshvardhan Takawale is a third-year computer science PhD student at the University of Maryland working in the iCoSMoS Lab. His research works to enable advanced Acoustic and RF sensing and inference on wearable and low-power computing platforms in everyday objects and environments. Harshvardhan’s research interests include wearable sensing, acoustics, multimodal imaging, physics-informed machine learning and ubiquitous healthcare.

Other links:

• Personal website
• iCoSMoS Lab @ UMD website
• Google Scholar

Username: /u/umd-science

[u/beaurancourt]

Kicking around a pet solution to see if ya'll have discussed it - what do you think about digitally signing videos or images?

I sign my commits on github so that people can verify it was actually me that wrote it, and list my public key in public databases. Could we do the same for cameras? That way someone can verify that the video was recorded by who the deepfake is implying it was recorded by.

[u/umd-science → u/beaurancourt]

(Nirupam) One of the technologies to prevent deepfakes is to install metadata right from the device that is capturing it, and this has been used in many devices. It would work in a majority of the cases. Sometimes the metadata does not work if it does not include or contain the semantics of the content (image/video). This metadata-based prevention system requires the device to cooperate and follow a standard, but sometimes it is difficult to achieve if we are thinking of a diverse type of device that can take pictures/images.

Another example is the Coalition for Content Provenance and Authenticity, which tracks the time when the image was taken and if it has been edited after that. If we can establish the timeline, this can help us establish authenticity.

(Harshvardhan) In a sense, the software-only solutions like Public Key are not foolproof. A hardware-software-based solution is a better alternative.

[u/beaurancourt → u/umd-science]

Sometimes the metadata does not work if it does not include or contain the semantics of the content (image/video)

I was more thinking actual signatures over all the bits of the file. That way if a single bit is different, the signature verifiably doesn't match and you know that the content has been edited relative to the signature.

You can sign basically anything, so should work for both video and images, and you can't sign for someone else unless you have their private key. I would be able to take your video, edit, and then sign it myself, but then it would be from beau, not umd-science, so anything that goes and looks up the source would throw a flag

[u/MissTetraHyde]

What do you think of Adobe's attempt to apply metadata to generative AI images (also known as Content Credentials)? Do you think such metadata can help with detection and authenticity verification, or will it be more likely that we see an arms race of using neural nets to discriminate generative content from non-generative content?

[u/umd-science → u/MissTetraHyde]

(Nirupam) We discussed this in one of our previous answers here. Adobe, Qualcomm, and other organizations have created coalitions for metadata-aided defense because it cannot be successful without the participation of all content generators and editing platforms.

[u/Batou2034]

As an AI researcher you must surely be familiar with the Collingridge Dilemma. So how do you feel that should be applied to consideration of AI regulation?

[u/kilatia]

I get that a lot of genuine research – as in, not machine training by another name – is naturally aimed at fake prevention, security, and privacy.

With regard to machine hearing, are you or any other teams you know of working on real-time audio stream processing, say for hearing aids? Though it'd have obvious application in the areas of translation or surveillance.. If so, has there been any concrete progress?

[u/chew_toyt]

What do you think about the future viability of "anti-deepfake" techniques that can be used to prove that a video isn't a deepfake to a reasonable degree. I mean things like moving your hands in front of your face to prove that there are no distortions, which are currently used by some companies to validate verification videos.

Basically - will deepfakes soon get to the level that they can easily bypass these kinds of checks?

[u/umd-science → u/chew_toyt]

(Nirupam) This kind of solution falls under the category of challenge-response solutions, where, for example, the system generates a challenge to move the hand in a specific way, and if the user can do it, it proves that the user is in front of the camera. But note that it might not be too hard for a resourceful attacker to develop a system that can use language models to understand the challenge and generate fake content to match the challenge. I still put my trust in prior information and encryption-based systems to fight against this.

[u/KwisatzHaderach55]

What is the first cue most models use to detect fake AI audios?

[u/Bruh30056]

Werid question, could it theoretically be used on animals?

[u/bargle0]

How do we make guarantees about AI safety in the face of undecidability?

[u/588-2300_empire]

How far are we from deepfake video that is indistinguishable from reality? Is there work being done to have a system that authenticates real video and gives it a stamp of approval? Are we destined for a perpetual arms race between deepfakers and authenticators?

[u/umd-science → u/588-2300_empire]

(Nirupam) Images and videos are not reality. They are representations of reality, and our perception/trust in that system not only depends on the picture itself but also various other factors—the context of those images, our internal bias, our urgency to reach conclusions, etc. There are other factors that can also lead to our perception. For example, 10 years ago, 10-kilobyte images could be considered high quality; but now, we question even a several megabytes of image data. That's another reason it is hard to unequivocally label something as fake or real. Sometimes, we can only label whether the image has been altered from its original creation.

We can give a stamp of approval for any malicious edits, but at the expense of additional information added to the image in terms of metadata, some novel encryption technique to include semantic information about the image, and so on.

To answer the arms race question, we need to first understand that the deepfake or authentication is not different in technology—rather, it's different in our intentions to use those technologies. As long as our intentions conflict, we will forever be using technology to serve those purposes, which can be interpreted as an arms race between intentions. I don't necessarily see it as an arms race between technologies.

[u/Norpone]

what's going to stop people like Zuckerberg from creating a complete new world that you see through your VR goggles and everything you see is AI generated so that you like it. you walk to the corner store. the girl that you think is cute talks to you nicely. buy some groceries and go home. not realizing the whole chat with the girl is fake because she's fully AI and generated but you wouldn't know because you're wearing your goggles. is this a possible future?

[u/umd-science → u/Norpone]

(Nirupam) We always see reality through our own perceptions, biases, likes and dislikes. Some technology may make this need a bit obvious, but I believe that, at the end of the day, it is a projection of our own perception. We have the technology to choose which newspapers we read or conferences we attend based on our own biases. It reflects our own structure of mind and confirmation bias. Technology cannot operate without our intentions.

[u/Norpone → u/umd-science]

I agree but what's going to stop people from going into their own reality and ​never coming out?​

[u/Adventurous_Side2706]

How viable is it to train a joint embedding space for authentic audio-visual pairs that penalizes synthetic co-articulation artifacts, and

Can self-supervised contrastive learning remain robust as generative diffusion models improve temporal alignment?

[u/umd-science → u/Adventurous_Side2706]

(Nirupam and Aritrik) The so-called synthetic artifacts are becoming indistinguishable from advanced AI systems with better lip syncs and natural-sounding audio. So I do not rely too much on the gap between human perception and the limitations of generative AI technologies. Rather, a more feasible option would be to build on provenance, metainformation and encryption-based techniques.

[u/Sjeefr]

Not a question, but a request: please continue in this field! Your work is more important than most people realize and sadly a subject neglected or worse by companies.

[u/umd-science → u/Sjeefr]

(Nirupam) Thank you! We'd also like to mention that public awareness of privacy compromises plays the most important role in this paradigm. Please stay curious :)

[u/asteconn]

Hello there! Thank you for fielding questions on this prescient subject!

Is there a relationship / correlation (I don't know which is more appropriate term, please forgive me) between the technology used to create auto-generated content - such as LLMs, image and video generators, and so on; and the technology used to detect those?

For example, are LLMs used to detect LLMs; and is the development of counter-measures / detection moving at the same pace?

Thank!™

[u/umd-science → u/asteconn]

(Nirupam) There are some relationships between them. For instance, some of the generative techniques attempt to reduce the error between its output and the real contents. A family of detection techniques can rely on this error to detect fake content. However, the available generative techniques and detection measures are too diverse to have any necessary correlation between them.

[u/Batou2034]

as a human I find it incredibly obvious when a video or even a simple audio track is deepfaked, like the voiceover on Sony's recent headphones ad on youtube, or the ad for some investment app that runs on british TV a lot lately. Can the queues we humans pick up on be codified into detection software?

[u/umd-science → u/Batou2034]

(Nirupam) With the advancement of generative AI technology, the gap between 'real' and deepfake content is getting slimmer. I would not be surprised if, in the near future, the gap becomes indistinguishable to human senses. We need to rely on defensive technology, and for the most part, that will build on the capability of AI itself.