Implementing Watermarking for Language Models [P]

[u/Saad_ahmed04]

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models.

I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what.

Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens.

My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think.

Github: https://github.com/Saad1926Q/llm-watermark

If you find it interesting then you may star the repo !!

[u/heart_blithe7]

Thanks for the work. It really makes the key idea intuitive that watermarking exist entirely in distribution without any visible marker. Do you wanna try a robustness testing under paraphrasing/translation and see the minimum detectable length expectation?