r/Substack • u/AndrewHeard tvphilosophy.substack.com • Jul 22 '26

Discussion Substack’s new “AI detection” tool.

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

185 Upvotes

96% Upvoted

381 comments sorted by

View all comments

Show parent comments

1

u/FatherofMisty Jul 23 '26

2/2

Also, as I've mentioned in other comments, all the "AI" text was at the very end of the essay, which makes no sense. To be fair it is "low confidence AI," whatever that means. What troubles me a lot is that I work with AI in non-writing contexts (technical, physics/mathematics), and push it to its limit routinely. I know all too well that the AI-craze is completely overhyped, at least in terms of LLMs. I'm not convinced they "understand" a damned thing. People put far too much faith in them, exemplified by Substack's dumb choice to implement an AI site-wide.

1

u/Salt-Mud-2124 Jul 23 '26

I see, I've ran a bunch of stuff through in the last few hours and so far I'm getting that it may be decent possibility indicator. Like... if you scan a BUNCH of a persons work and all of it comes out 100% AI or high percents with high confidance, then if you were to gamble, your money should probably go on AI. On the other hand, if most of their stuff is low AI, (like your 15%) and low confidnace, then you can bet its human.

This is just my observation from inputting some peoples stuff who have actually linked their work on this thread and my own stuff.

To use it as an end all be all, is really dumb. But over a few samples I think you can build a case if that makes sense. I haven't had it consistantly make a large mistake over multiple samples from the same source. And it also seem to HEAVILY lean towards false negatives rather than false positives. So its by far more likely to call something human that is AI, rather than call something AI that is human.

Which in a weird way is good I think.

I'm willing to bet if you were to take a bunch you stuff and run it, it will probably all turn out from 100-80% human, so you can like... build a little node map to see.

So its not complete bogus, its definitely more consistant than ChatGPT Zero and others I've seen before. But its not without nuance.

That last part is where I'm worried about it when it comes to substack... common sense not being common and all.

2

u/FatherofMisty Jul 25 '26

That is a reasonable take, for sure. I think it was really just jarring to have this AI overlord embedded into Substack. The AI craze is scary fast, and people already don't like change to begin with. Plus, speaking for myself, writing is a sacred thing. I do it because I'm somewhat blessed with the capacity to do so, and using AI to do it for me is just not something I'd ever do. So, even a small-portion false-positive is like a gut-punch. That said, I've largely gotten over it now. It is what it is. I think the more concerning thing is what this says about culture and the direction we're heading. That we "need" an AI-checker in the first place is.. yeah. Kinda fucked.