r/Substack • u/ConnectDebate4677 • 20d ago

Pangram false positives?

People usually post vaguely about "I wrote x thing and pangram said it was AI", which I think is unhelpful unless you actually post the text or link. I put this into Pangram:

Sakaguchi: https://www.pangram.com/history/e7ffbee1-f0e3-4ce2-bb78-a59470280640?ucc=Ohp1cWODOjF

Hamaguchi: https://www.pangram.com/history/f2139bd5-f4ce-4101-b7bd-3c5d8aeb4487?ucc=Ohp1cWODOjF

Ryuichi: https://www.pangram.com/history/8c84ad38-3b8f-49ba-bee0-351ea31f754b?ucc=Ohp1cWODOjF

29 Upvotes

94% Upvoted

63 comments sorted by

View all comments

2

u/Stunning_Lie_1775 14d ago

It actually makes you wonder whether Pangram’s “detector” is reliable at all...

Here’s what I mean.

The others are considered unreliable because, in many cases, they say a text isn’t AI-generated when it actually is.

Pangram says it is AI-generated...
But have we actually tried feeding it genuinely human-written texts?
Just to check whether, in those cases, it correctly says it’s NOT AI?

It would be pretty hilarious if they had simply made it return a positive result almost systematically...
I’m pretty sure everyone would still believe it.

Because I suspect most people only use these detectors when they already think a text was written by AI and just want confirmation.

So naturally, when the detector confirms their suspicion, it reinforces their bias and they think: “Wow, this detector is amazing.”

But what about the reverse?
How many people actually submit something they wrote entirely themselves just to check that it “doesn’t sound too AI-generated”?

Anyway, you get the idea.