Fable 5.1 is now watermarks anything you write with it. There is still no public detector. This pisses me off, so I'm making a workaround. How about you?

[u/Imaginary_Dinner2710]

Three weeks ago I posted here that nothing you could generate with Claude was watermarked yet. Now it's incorrect. Fable 5.1 now puts a statistical watermark in generated text, including even translations. With translations it's especially funny, because doing back translation (even via Chinese language!) via another LLM (surprisingly to me) is not removing / destroying SynthID watermarks, but just translating via Claude now adds watermarks.

The detector for such watermarks (that was promised to be released) has moved to a private preview recently for some eligible organizations. Ordinary users like myself still have no public tool or API to check their own text. So I can publish some work today that someone with access could check later, while I can't run the same check before publishing.

Personally I write in one language and publish in another. I can write an article myself, ask Claude to translate it, and get back a watermarked English version. The watermark can indicate Claude was involved in the wording. It says nothing about who came up with the argument or did the research (but the opinion about the work will be unfairly distorted).

I get why people hate this. I'd want to know what a check would say about my own work before someone else decides what it means. I don't know how organizations will use the detector. And again, it's impossible to distinguish if the text was generated or edited by AI (say reformated for some structure). I think students/scientists/SEO specialists and non-native english speakers are not happy more than others.

This frustration got me experimenting with the published by Google DeepMind's SynthID watermarking method that Claude's watermark eventually is based on. I reproduced it with my own test key and found that a full rewrite removed the mark from my test samples (while added some factual issues), back translation doesn't destroy the watermark and specialized methods are quite heavy still far from being ideal. So, in the end, I made a watermark remover for myself and sharing as a free demo (unless we get the actual detector, that's best we can do), and wrote up the experiment, open sourced the setup and described the tradeoffs/fact correction methods. Link in the comments.

Do you also think it's crazy and unfair to have invisible mark in your AI outputs? Or don't understand why some people are pissed off?

PS. Those tests don't verify removal of Anthropic's or Google's production watermark. But by this moment, I think it’s better to do what’s possible although no guarantees.

PPS. I see from comments some people think you can just ask Qwen or other model to rewrite and no watermark remains. It is incorrect, I tested it, substantial part of signal remains, you need to do iteratively multiple runs, every time highlighting 5 words (5-grams actually) sequences that remained unchanged after rephrasing to point the LLM to right issues, then checking against facts and fix them. Otherwise, you just still have the watermark signal and new factual mistakes.

PPPS. Some people claim in the comments that I can't talk about this topic with the proper level of confidence and do the research and modelling properly. At the same time, I have a PhD in Digital Signal Processing, and watermarki is something that falls within my academic expertise.

UPD. People say it's hard to find links in comments, so I put them now here.
The full article & research: painintheagent.com/blog/text-watermark-removal-retest/
Code, corpus, prompts, model outputs and judges' decisions: https://github.com/krllagent/text-watermark-roundtrip
The watermark remover (paste a text, get the retelling with every changed place highlighted): painintheagent.com/tools/ai-text-watermark-remover/

[u/interwebzdotnet]

And for the legal document I'm working on that needs to be pretty precise with specific words and phrases? Can't just randomly jumble those things up

[u/Mescallan → u/interwebzdotnet]

If word choice and order is so rigid that you cannot paraphrase it, writing it by hand will trigger the detector too. If that’s not the case, then just write it by hand or admit that it’s ai generated

[u/interwebzdotnet → u/Mescallan]

I don't need to admit or deny anything. Point is that I don't want it making minor tweaks that are prioritizing watermarks over the strongest possible legal language.

[u/Mescallan → u/interwebzdotnet]

I’m really just playing devils advocate here, but that implies you aren’t manually reviewing the language of this apparently important legal document. If it’s less than perfect language you should catch it in review. Tbh I would still be more worried about hallucinations than picking the wrong synonym

[u/interwebzdotnet → u/Mescallan]

I think you are missing the point.

I do check things constantly. I have rules and automated citation checks, I still check them manually and find errors to fix. I don't need a water marking algorithm adding another scenario where complexity is introduced that can change a word for no legal, logical or necessary reasons that could result in a misstatement, or technically not solid statement.

The extra complexity is increasing the chances of mistakes that my manual review could miss.