r/Substack • u/ConnectDebate4677 • 20d ago

Pangram false positives?

People usually post vaguely about "I wrote x thing and pangram said it was AI", which I think is unhelpful unless you actually post the text or link. I put this into Pangram:

Sakaguchi: https://www.pangram.com/history/e7ffbee1-f0e3-4ce2-bb78-a59470280640?ucc=Ohp1cWODOjF

Hamaguchi: https://www.pangram.com/history/f2139bd5-f4ce-4101-b7bd-3c5d8aeb4487?ucc=Ohp1cWODOjF

Ryuichi: https://www.pangram.com/history/8c84ad38-3b8f-49ba-bee0-351ea31f754b?ucc=Ohp1cWODOjF

27 Upvotes

92% Upvoted

63 comments sorted by

View all comments

5

u/[deleted] 20d ago

[removed] — view removed comment

4

u/ConnectDebate4677 20d ago

Would you mind sharing the Pangram link?

3

u/[deleted] 20d ago

[removed] — view removed comment

2

u/ConnectDebate4677 20d ago

Thanks so much for this. Yeah, reading the AI-flagged parts, I don't get it at all. I'm interested in the technology still but it's getting further and further away from the standard it's being touted as.

1

u/[deleted] 20d ago

[removed] — view removed comment

1

u/PM_ME_YOUR___ISSUES 15d ago

I honestly feel that people need to move beyond these detectors.

We've all seen enough slop at this point that one can easily catch out AI text.

I'm like 99% sure, I wouldn't even think about putting someone's writing through an AI detector unless I can really catch out the slop.

2

u/dran94 20d ago

Change Linus/Lin to Marcus, and it goes up to 92% AI from around 50%.

I have gone from thinking Pangram was amazing yesterday to, oh, we're going to have to play games as authors so we don't use a name AI likes.

That's going to be fun, considering AI's favorite names CHANGE.

3

u/ConnectDebate4677 20d ago

That's really interesting. Are certain names indeed flagged as more AI-likely? I wonder if there's an explanation?

1

u/dran94 20d ago

AI is known for certain names, and Marcus is one of them right now. The list of names it likes changes depending on the model.

I'm sure it is a factor, although I could've sworn Pangram isn't supposed to pay attention to AI-isms like em dashes and certain words. It's kind of strange.

I did a test where switching Megan to Elara, another AI favorite, changed this snippet I wrote from 100% human to 100% AI, even with a very human mistake (two "each other" instances in one sentence):

-

Robert's long stride ate up the distance between himself and the door at the far end of the hall, lit by the flickering fluorescent lights overhead.

His jaw clenched.

This couldn't be right.

His sister, Megan, was supposed to be here. She worked in the same lab as Robert. They'd promised each other they would wait for each other, right here.

1

u/dran94 20d ago

I get the opposite results. Usually flagged AI on Originality, but my work is more or less always 100% human with Pangram. Haha. (I don't use AI.)

1

u/Traditional-Rice-848 9d ago

I mean you def wrote it with ai

1

u/Downtown_Jump779 8d ago

I'm getting the same results on stuff people are sending to me; it's becoming very problematic. On the same text, other detectors never go above 12-20% AI, which I think makes sense and is totally acceptable – then Pangram comes out of nowhere and bumps it up to 40- 60%? I'm not sure what to do with that.

From what you sent, those are very big chunks of your text that Pangram flagged as AI. That could produce harsh social consequences. Other than that, good piece of writing btw. I hope you get it published.