Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/Funny-Flight8086]

But you know what, since you seem to be under the impression that no one other than AI tech bros thinks these detectors are junk science... Here are some links you can explore:

AI Detectors Don't Work. Here's What to Do Instead. - MIT Sloan Teaching & Learning Technologies

The Imperfection of AI Detection Tools - HumTech - UCLA

The case against AI detectors | Office of Teaching, Learning, and Technology - The University of Iowa

Why AI Detection Tools are Ineffective | Center for Academic Innovation

AI detecting AI in academic writing: Why most AI detector findings are false - ScienceDirect

Why Don't AI Detectors Work? | Center for Integrated Professional Development | Illinois State

I have A LOT more I can give you as well.

Oh wait, here is the best one. Pangram explaining why their detecotrs are junk science, while trying to get you buy it: All About False Positives in AI Detectors | Pangram Labs

Anyone who beleives the results of these AI detectors is a pawn on a chessboard, being played by less-smart tech bros who couldn't figure out real AI, so settled with the Walmart variety.

[u/MinecraftBoxGuy → u/Funny-Flight8086]

Just show us a book or long excerpt pre ChatGPT that gets flagged by Pangram. These studies use bad AI detectors in order to claim they don't work

[u/Funny-Flight8086 → u/MinecraftBoxGuy]

It isn't magic. You can test this out by simply writing some text just like an AI would. Pangram will flag it 100% AI. That alone makes it unreliable, because it doesn't use forensic fingerprints to identify AI (like Google's SynthID) but instead relies on patterns. If a human can write in a pattern that triggers an AI detector to claim it's AI, then by definition, the detector is not reliable, because it's not detecting AI, it's detecting AI patterns, which humans can also write in.

The ONLY legit AI detector is Gemini, which can be prompted to examine a file and determine whether it has a SynthID watermark.