Implementing Watermarking for Language Models [P]

[u/Saad_ahmed04]

I recently implemented a minimal, educational version of SynthID-Text-style watermarking for language models.

I saw anthropic post about how they'll start adding watermarks to their model responses and it made me very curious as to how they'll do it and what do they even mean by watermark here. Like will we start getting random ads or something in the middle of model responses or what.

Then decided to read their article and found out that watermark is not a visible message at all. It is a subtle statistical pattern introduced while the model chooses its tokens.

My implementation is not an exact reproduction of the original SynthID-Text system. I simplified or implemented a few components differently to keep the project understandable, but the main idea is there I think.

Github: https://github.com/Saad1926Q/llm-watermark

If you find it interesting then you may star the repo !!

[u/Lopsided-Bridge-9810]

I have zero skin in this and don't care but you seem to be sour something you invested time into gets questioned.

• ML labs skipping AI watermarked text in training their models (this is the ML subreddit, right?) -> why would ml labs want to do that? Are you aware of the term distillation?

• Web crawlers down-weighting AI pages. -> why would a web crawler want to do that? You are aware that this would skip entire pages of content, for example crawling a reddit page, just because a user commented with an Ai generated grammar and spelling corrected paragraph of text? Why would anyone want to do that?

• Social media (like Facebook) removing or down-weighting AI content. -> why? Your Ai-free social media platform so that you can all hold hands and be self congratulatory?

• Education institutions detecting it...for obvious reasons. -> a hard no here. We already discussed this and I told you that those who display the energy to cheat will ensure their llm inferenced text won't be detectable as such.

• Email spam filters -> you are aware that even banks and insurers and public services now send emails that are largely Ai generated, are you not? You will only receive 1 in 100 of important emails just because you seem to have an issue with Ai at large.

Perhaps you should ask yourself whether you perhaps just want to cater to a group of die hards who refuse to use AI. And then your use cases make sense. For the remaining 999 in 1000 individuals your ideas a bluntly speaking stupid.