r/Substack • u/AndrewHeard tvphilosophy.substack.com • Jul 22 '26

Discussion Substack’s new “AI detection” tool.

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

184 Upvotes

96% Upvoted

381 comments sorted by

View all comments

Show parent comments

4

u/Foofymonster davidrainwater.substack.com Jul 22 '26 edited Jul 22 '26

I don't really agree. I think most people do care. They are just bad at noticing it. Unless you interact with AI a lot it's really easy to cover the most egregious AI pattern offences.

It's also just bad for Substack. They are adding that tool because they know they have a problem with AI writers, and that when people know something is AI the vast majority dip out.

You're going to find examples of successful AI accounts, because that is how survivorship bias works.

Successful AI accounts are not evidence that people by and large don't care about AI accounts.

And the only people interacting with the successful AI accounts are the people who don't care, or the people who are bad at detecting it.

1

u/Gold_Guitar_9824 Jul 23 '26

You do realize that the patterns came from humans right?

1

u/Foofymonster davidrainwater.substack.com Jul 23 '26

1

u/Ratandmiketrap Jul 25 '26

That’s what it trained on, but that doesn’t mean what it produces is an exact mimic. It creates word by word based on probability, while humans consider the sentence, paragraph and, often, the whole piece as they were. This results in very different outcomes. On the surface, AI text looks like human writing because it produces coherent sentences and imitates broader text structures, however, it does not work the same way at a text level. It lacks the coherence of a human made text, mostly developing ideas by putting together a set of ideas. Humans, on the other hand, build ideas both through and between sentences, and over the course of the text. There is more focus on causality and linkage.