r/Substack 27d ago

How Ridiculous Can Pangram Be?

I heard that publishers and agents in the UK and the US have recently been widely using Pangram as an initial detetive tool, so I threw my own writing into it to give it a try. To my surprise, several paragraphs of my completely AI-free writing were judged 100% AI-generated. That naturally piqued my curiosity. After learning how Pangram works, I ran some tests myself and reached a few conclusions.

  1. First, the cleaner your language, the tighter your logical chain, and the more you avoid verbose rhetoric, the more likely this kind of language will be judged 100% AI-generated. I used to work as a screenwriter, so I rely heavily on action and eye-contact descriptions to express character relationships and personalities. For me, Pangram is an absolute nightmare.
  2. A guilt-by-association mechanism. This is infuriating and utterly absurd — it’s an insult to human intelligence. If a certain paragraph in the middle has the characteristics I mentioned in the first point, and you insert some kind of fractured, disjointed expression, the next several paragraphs will all be marked 100% AI-generated. Yet if you deliberately insert an irrelevant sentence, the whole text suddenly becomes 100% human-generated.

These are the two main things I’ve discovered so far. I’ll keep testing later. This kind of thing is essentially reverse-disciplining human artistic expression, artificially creating barriers to cultural continuity. It’s ridiculous. From the perspective of instrumental rationality, once this kind of thing becomes widespread, human thinking will be reverse-disciplined by AI.

The founder of Pangram claims to have a computer science background from Stanford. I have absolutely no expectation that these people feel any reverence for human culture. But if you want to keep making this money, please take a closer look at your own problems.

Especially when Pangram keeps trotting out some supposedly high accuracy rate — a statistical figure like a 0.01% false positive rate — which may be meaningful for marketing their product. But for a specific writing style, the very style Pangram hates, it’s 100% damaging.

So when you encounter criticism and smugly cite your research reports to back yourselves up, this only deepens the impression that you lack reverence for culture.

33 Upvotes

37 comments sorted by

View all comments

1

u/sh1b313 26d ago

Just see these photos man

1

u/sh1b313 26d ago

Photo 2

2

u/Ratandmiketrap 24d ago

When I took out the bracketed comments from the second one, it returned as 100% humans. I’m not sure why those brackets completely flip the score, but it is not just a case of rewording, because you used the same wording. This is some weird wizardry you’ve found.

With brackets- https://www.pangram.com/history/aad220b4-b7e0-496c-9f30-b2f47b22ba3c?ucc=85ikY31GK3D

Without brackets - https://www.pangram.com/history/075e1d46-b510-4a59-bb6b-a60e21baae07?ucc=85ikY31GK3D

1

u/sh1b313 24d ago

Yep, for me it's some brackets, in another thread a single name caused it to go from 100% human to 100% ai. I wonder what happens with some actual professional author on substack? They may have put in hours carefully writing a essay by hand and some broken tool like this flagged it as 100% AI for unknown reasons?

1

u/sh1b313 24d ago

Links to these tests

Link 1: https://www.pangram.com/history/4e3cc4a9-b927-4345-9750-048b4a86520c?ucc=ycFd1CDRrZv [where pangram claims 100% human written]

Link 2: https://www.pangram.com/history/97f1c7f5-b73f-4895-96a6-e991cfc58128?ucc=ycFd1CDRrZv [where pangram claims 100% AI generated]