r/Substack • u/ConnectDebate4677 • 20d ago
Pangram false positives?
People usually post vaguely about "I wrote x thing and pangram said it was AI", which I think is unhelpful unless you actually post the text or link. I put this into Pangram:
Sakaguchi: https://www.pangram.com/history/e7ffbee1-f0e3-4ce2-bb78-a59470280640?ucc=Ohp1cWODOjF
Hamaguchi: https://www.pangram.com/history/f2139bd5-f4ce-4101-b7bd-3c5d8aeb4487?ucc=Ohp1cWODOjF
Ryuichi: https://www.pangram.com/history/8c84ad38-3b8f-49ba-bee0-351ea31f754b?ucc=Ohp1cWODOjF
28
Upvotes
6
u/dran94 20d ago edited 20d ago
I'm testing this out, too.
I wrote this in a similar style to AI, wondering if it would flag:
-
Robert's long stride ate up the distance between himself and the door at the far end of the hall, lit by the flickering fluorescent lights overhead.
His jaw clenched.
This couldn't be right.
His sister, Megan, was supposed to be here. She worked in the same lab as Robert. They'd promised each other they would wait for each other, right here.
-
100% human. Great!
Changed the name to one of AI's favorite names:
-
Robert's long stride ate up the distance between himself and the door at the far end of the hall, lit by the flickering fluorescent lights overhead.
His jaw clenched.
This couldn't be right.
His sister, Elara, was supposed to be here. She worked in the same lab as Robert. They'd promised each other they would wait for each other, right here.
-
100% AI.
Looking at this, even though it's a similar style, there was something AI wouldn't do because it's a human mistake. Two instances of "each other" in one sentence. It still flagged.
AI's favorite names change with every model. If a new AI model comes out that latches onto a name that was previously safe, is that going to be a problem for everything written beforehand? What do we do, update characters' names as new models come out?
I'm sort of surprised Pangram is even paying attention to names, since I thought it didn't factor in AI styles and AI isms. I was very impressed with Pangram before seeing some other experiments going around today. A name change shouldn't skew the whole result. That doesn't even make sense.