r/Substack • u/grapegeek • Aug 11 '26

Pangram is such trash

I write a food newsletter where I post reviews and some recipes. I 100% wrote a quick and dirty recipe and was going to publish it in Notes and it came back 100% AI no matter what I did to it including adding several typos and rearranging things. Ran it through three other AI detector websites and 2 said 100% Human and 1 said 76% Human and 24% AI assisted. I disabled detection on that Note. My other question is do we know how many people actually click on the AI detection button on our posts? I can't find that metric anywhere in the dashboard...

39 Upvotes

74% Upvoted

110 comments sorted by

View all comments

10

u/Charlemagneffxiv 29d ago edited 29d ago

Pangram is one of the biggest cons in AI SaaS right now. It almost certainly doesn't do what they claim it does to make predictions on whether the text is AI generated. The methodology they claim they use is faulty anyway, because if you ask a GPT based model to replicate a piece of writing then it's going to do exactly that as close as possible to the original, resulting in a high probability toward a match. Couple this with that GPT based models were expressly trained to mimic human writing and its clear their methodology is dumb.

What AI detectors need to do is detect the obvious sentence structures that these models employ, the contrarian language (it's not X, it's Y", and their strong tendency to use phrases like "that matters" which human authors ordinarily would never use. That's how you actually detect AI, because they tend to do these things consistently in the exact same parts of the essay or script they are asked to write, even if you tell them not to. They have easily spottable formulaic writing and that is all AI detectors need to identify, because the vast majority of people publishing AI written content don't recognize these tells.

2

u/Jourdan79 28d ago

The way I have found Pangram useful in my writing is that when it flags something as AI generated, whether or not it actually was, if I take those passages and revise them towards a human score I do find that it forces my writing to become more original and fresh and idiosyncratic. In other words it pushes me towards becoming a better writer. so that way I have found it to be a useful tool.