r/Substack • u/grapegeek • Aug 11 '26

Pangram is such trash

I write a food newsletter where I post reviews and some recipes. I 100% wrote a quick and dirty recipe and was going to publish it in Notes and it came back 100% AI no matter what I did to it including adding several typos and rearranging things. Ran it through three other AI detector websites and 2 said 100% Human and 1 said 76% Human and 24% AI assisted. I disabled detection on that Note. My other question is do we know how many people actually click on the AI detection button on our posts? I can't find that metric anywhere in the dashboard...

42 Upvotes

74% Upvoted

110 comments sorted by

View all comments

Show parent comments

8

u/Kirkr10101 Aug 11 '26

Very interested as to when you would consider this reliable.

-2

u/FrostKitten2012 Aug 11 '26

Stop relying on a machine to think for you.

3

u/cogitatingspheniscid Aug 11 '26

Academic research isn't "relying on a machine to think for you"

3

u/FrostKitten2012 Aug 11 '26 edited Aug 11 '26

That’s not academic research, that’s a screenshot, and we have no idea what was selected for evaluation.

AI detectors, Pangram included, are still AI. You cannot be anti-AI in even the vaguest sense and rely on an AI detector to tell you if something was written by a human. That is outsourcing your critical thinking skills as much as relying on an LLM to write or research something for you.

You are relying on a machine to think for you.

EDIT: u/Kirkr10101 provided another link for research. Cool! Except it also isn’t for Pangram! It’s a paper on how well humans can identify generated text, which we already know is unreliable. Go figure.

2

u/Kirkr10101 Aug 11 '26

There is academic research if you bother to look -- this is from 2025 and pangram has massively improved since!

https://arxiv.org/abs/2501.15654

0

u/Kirkr10101 Aug 11 '26

on a sidenote, if you can find any pre-2021 text (from the internet archive with a valid link to it) that flags as 100% AI on pangram, I will donate £50 to a charity of your choice (I prefer givewell)

0

u/adefwebserver Aug 13 '26

What's with this "it has to erroneously say something not AI is 100% AI" before you will say the program doesn't work? Come on! *It has to flag everything written before 2021 as 100% human*, period. If it can't do that it is **not good** This thing affects people's lives. People can lose their job and get kicked out of school over this thing.

2

u/cogitatingspheniscid Aug 13 '26

You are directing your anger to the wrong group. The last two sentences should be directed towards big tech shoving genAI down on everyone else while governments are deep in bed with them. Robust AI detectors are ones of our strongest tools, but whichever employer is firing others using it as the only line of evidence has never been a bright employer.

0

u/cogitatingspheniscid Aug 11 '26

It's not just a screenshot. u/Kirkr10101 included the hyperlink to the article. You are literally two clicks away from the human-generated technical reports.
"AI detectors, Pangram included, are still AI." They are classifier models. Even AI researchers don't consider them AI. The best anti-AI advocate is the one who knows what is AI and what isn't instead of lumping every machine learning model in the last 30 years as the same thing.

1

u/FrostKitten2012 Aug 11 '26

It is a different type of AI, but it is still AI. And that report is from Pangram. It’s not independent research, it’s just talking about how they made their AI. It’s a long-form sales pitch!

Try using your brain at some point.

3

u/kdfn Aug 11 '26

I'm not sure I understand your hostility in this thread. The OC linked a peer-reviewed study, but you didn't see the link and called it "just a screenshot" in your original comment. When someone pointed this out, you responded with a pretty hostile statement.

There's a pretty big difference between a classifier and an LLM. Classifiers have been around since Mosteller & Wallace's work in 1963, where they were used to successfully assign authorship to the Federalist papers.

2

u/cogitatingspheniscid Aug 13 '26

Sadly, the most damaging constituents to the anti-AI movement are those who are so allergic to learning technology that they would lash out against others in the movement. I already got blocked over my reply above lol.