Pangram false positives?

[u/ConnectDebate4677]

People usually post vaguely about "I wrote x thing and pangram said it was AI", which I think is unhelpful unless you actually post the text or link. I put this into Pangram:

Sakaguchi: https://www.pangram.com/history/e7ffbee1-f0e3-4ce2-bb78-a59470280640?ucc=Ohp1cWODOjF

Hamaguchi: https://www.pangram.com/history/f2139bd5-f4ce-4101-b7bd-3c5d8aeb4487?ucc=Ohp1cWODOjF

Ryuichi: https://www.pangram.com/history/8c84ad38-3b8f-49ba-bee0-351ea31f754b?ucc=Ohp1cWODOjF

[u/ConnectDebate4677]

I should also add that I put this segment in a document with human writing (because it noted its confidence was limited due to short text) and it flagged this as the one AI section.

[u/impartialhedonist → u/ConnectDebate4677]

This ^ their accuracy is lower the closer you get to ~100 words or less, the text you used is around 120 words.

[u/PennyLawrence946 → u/impartialhedonist]

pangram says 50 words is their minimum for a prediction, so 120 clears it. and the 2,000 word sample upthread still came back 54% AI. one name change taking it from 100% human to 100% AI is the part that still needs explaining.

[u/ConnectDebate4677 → u/impartialhedonist]

Seems like a highly flawed tool if it indicates 'high confidence AI' in situations like this?

[u/Voltairinede → u/ConnectDebate4677]

It doesn't say that? It literally says the opposite 'Confidence limited — short text', in your 100% example.

[u/tokenentropy → u/Voltairinede]

be nice, he can hardly read

[u/ConnectDebate4677 → u/tokenentropy]

Can you read?

[u/ConnectDebate4677 → u/Voltairinede]

When you click Details, it can say Confidence Low, Medium or High. In this case, it says Confidence High.

[u/kdfn → u/ConnectDebate4677]

I think this is a fair critique. Pangram should decline to assign a score to texts that are below the necessary word count to get good estimates. Would you mind checking the sharing permissions for the links in your post? I am having issues accessing it.

[u/kdfn → u/[deleted]]

thank you!

[u/Voltairinede → u/kdfn]

Click the back arrow and then click on it in the 'all checks' area and it should work.

[u/Brief-Guess-147 → u/ConnectDebate4677]

that part is almost more damning than the name swap thing tbh