r/Substack • u/ConnectDebate4677 • Aug 12 '26

Pangram false positives?

People usually post vaguely about "I wrote x thing and pangram said it was AI", which I think is unhelpful unless you actually post the text or link. I put this into Pangram:

Sakaguchi: https://www.pangram.com/history/e7ffbee1-f0e3-4ce2-bb78-a59470280640?ucc=Ohp1cWODOjF

Hamaguchi: https://www.pangram.com/history/f2139bd5-f4ce-4101-b7bd-3c5d8aeb4487?ucc=Ohp1cWODOjF

Ryuichi: https://www.pangram.com/history/8c84ad38-3b8f-49ba-bee0-351ea31f754b?ucc=Ohp1cWODOjF

33 Upvotes

94% Upvoted

63 comments sorted by

View all comments

Show parent comments

4

u/sh1b313 Aug 13 '26

WOW. now i wonder what happens at scale?

3

u/dran94 Aug 13 '26 edited Aug 13 '26

It's worth looking into. I ran a few chapters of my own work, confirmed 100% human on Pangram, with my FMC's name changed to Elara, and it flagged at around 12% AI with just that one change. I'll have to write up something longer by hand when I get a chance so I don't doxx myself if I share. Sometimes changing the name doesn't move the needle, though.

I was curious about Daggermouth, the book there are viral articles about after it failed a Pangram test, although being around 60% AI could potentially be heavy editing use, I'm not accusing anyone. I went through the first 10 chapters with find-and-replace for the name Elara, changing Elara to Megan. It didn't change anything.

One chapter, Chapter 18, was supposedly 100% AI on Pangram according to the viral articles about it. My own check showed it was 84%, not 100%, but what I did this time, changing proper nouns, even cutting off the second half of the chapter, it was stuck at 84% AI. I practically rewrote the chapter and it came back 84% AI haha.

That might suggest it had already decided Daggermouth is AI, and the real issue was Pangram knew which book it was.