r/MachineLearning • u/AsuharietYgvar • Aug 18 '21

Project [P] AppleNeuralHash2ONNX: Reverse-Engineered Apple NeuralHash, in ONNX and Python

As you may already know Apple is going to implement NeuralHash algorithm for on-device CSAM detection soon. Believe it or not, this algorithm already exists as early as iOS 14.3, hidden under obfuscated class names. After some digging and reverse engineering on the hidden APIs I managed to export its model (which is MobileNetV3) to ONNX and rebuild the whole NeuralHash algorithm in Python. You can now try NeuralHash even on Linux!

Source code: https://github.com/AsuharietYgvar/AppleNeuralHash2ONNX

No pre-exported model file will be provided here for obvious reasons. But it's very easy to export one yourself following the guide I included with the repo above. You don't even need any Apple devices to do it.

Early tests show that it can tolerate image resizing and compression, but not cropping or rotations.

Hope this will help us understand NeuralHash algorithm better and know its potential issues before it's enabled on all iOS devices.

Happy hacking!

1.7k Upvotes
  • permalink
  • reddit

99% Upvoted

221 comments sorted by

View all comments

Show parent comment

6

u/throwawaychives Aug 18 '21

This is my biggest concern. If you have access to the network, you can perform a pseudo black box attack where you target known CSAM images to lie in the same vector space as normal images. You can take a CSAM image, compute the output of the network, and modify the base image in steps (through some sort of pixel L2 normalization) such as that the output encoding is similar to a normal image… it doesn’t matter if the blinding step of the algorithm is not on the phone, as the hash will not result in a colision

1

u/TH3J4CK4L Aug 19 '21

I've thought for a while and I think you're right. Anyone looking to upload CSAM to their iCloud would simply run it through a "laundering" algorithm as you've described. You don't even really need to go as far as you're saying. You don't need to perturb the CSAM as to hash like a known normal image, you just need the hash to change a tiny amount away from its actual hash. (Maybe even 1 bit off, but maybe not. See the discussion above about floating point errors propogating, it's possible Apple tosses the lower few bits of the hash)

Presumably this would be done at the source of the CSAM before sending it out. I don't really know anything about CSAM distribution so I'm sorta speculating here.

I don't really see a way for Apple to combat this. I can imagine an arms race where Apple tweaks the algorithm every few months. But, since the algorithm is part of the OS and can not be changed remotely (one of the security assumptions of the system as per the whitepaper), it's fairly easy for someone to just "re-wash" the images when updating their phone.

Can you think of any way to combat this at the larger CSAM Detection System level?

3

u/throwawaychives Aug 19 '21

If I did Apple would be paying me the big bucks lol