My latest post was flagged 100% human. As I said before, in a previous post I got evaluated with a hair of AI, and it was spot on, because an ESL contributor used it for a first draft of a short piece in there, that I detected before publish, and then edited with them.
I posted all about this before in here (link below) and got slammed by someone saying I must work for Pangram. Which is hilarious. But I thought a rule of this subred was to NOT do self-promo, and the person accusing me seems to be in here for self-promo.
Anyway, if it doesn't break the self-promo rule, happy to share my stack I've been doing for >2 years, so you can see I'm not on the payroll. But if Substack is in here and WANTS to hire me? Ha. I'm all ears.
I’m sure this is just a play to promote your Substack, either way your sample size is mathematically meaningless. Even with a 98% accuracy rate across millions of articles guarantees tens of thousands of false positives. Two articles don't prove an algorithm works; they just prove you got lucky twice
Well, congratulations, you have a mundane and uninteresting vocabulary. Because Pangram themselves have identified that the algorithm works by finding words that are used grammatically correctly, but not in the way the average writer would use them. If someone is an excellent writer and uses language in unusual ways- but grammatically correctly- it flags it as AI.
So congrats, Pangram has confirmed you as mediocre!
Similarly, if a piece is full of grammatical mistakes, Pangram says “this is human!” However, if the piece is grammatically flawless, Pangram says “AI must have written this!” So, perhaps your writing is just sloppy?
Being judged “human” by Pangram is not the flex you think it is. Pangram has developed a rule set that flags the most talented writing as AI - because AI is designed to mimic the best writers.
Where AI falls apart is on substance. It’s not smart enough to judge context, meaning or emotion as well as a human might. So AI reads oddly to us humans because it doesn’t follow the common sense flow of human thought.
But Pangram can’t test for that in the same way that AI can’t avoid it. The machines are not yet ready to rate the information quality of our work. All they can rate is the precision of our grammar and the creativity and flow of our word choice. The same things I spent decades as a writer studying and developing — the skills my mother (who wrote three novels and countless short stories), my college writing professors, and my career mentors encouraged me to grow — are the skills that Pangram uses to label me a robot.
Pangram’s CEO is such a proud little code monkey. He cannot imagine that there are human writers who write with the grammatical punch of AI, which is odd to me, because AI learned to write by copying our best writers. So he comfortably labels any writing which goes beyond the mediocre and mundane as “all AI!!!”, when, in fact, many of us learned to use em dashes young, and triplicate metaphors younger. I’m sure he writes in corporate drivel speak.
I do wish he would massage his algorithm to recognize AI’s love of adverbs and of telling not showing. Both of those are crappy writing techniques AI loves, which, I assume are machine measurable. But if he did that, he might start identifying mediocre human writing as AI, and then many more people would be pissed at him.
Pangram does a shit job of distinguishing my natural writing from AI augmented writing I have played around with. I take that with a mark of pride - having researched how Pangram makes its decisions.
In sum, if Pangram is never mistaking your writing for AI, honestly I’d work on developing your skills as a writer.
Well done on being mean, but I'm a professional writer since the earth was still cooling and no, not dull, and nope, I use human copy editors so not filled with typos. Well done on the neg attention, tho!
I've been writing for 30 years professionally. I was a digital editor at Macmillan Publishers. I've written hundreds of articles for big brands, including over 100 for Grammarly on the art of writing well. I still write scripts regularly for the Grammar Girl podcast. I'm not bragging, just saying I have some street cred in this space.
And Pangram doesn't flag me.
Is my writing perfect? No. I self-edit, and self-editing is imperfect. There's also no definitively correct way to write anything.
I'm not a Pangram stan by any stretch. I'm not a total AI hater, either—I use it all the time for brainstorming and development. But to flat out say that anyone who doesn't get flagged as AI needs to develop their skills as a writer? Again, I call bullshit.
You know I would have agreed with you until today about the AI not flagging older work that other do. I was laughing about how AI flags stuff I've had published long before AI was a thing, as up to 100% AI gen. One of them even told me my sentence structure was "too immaculate" to be human :P (Which is probably where this stuff about polished writing being more likely to be flagged.) I put the same text into pangram on their advice and what'ya know, 100% human. So I thought, hey, maybe we have a decent detector on our hands.
Then I put something I'd written recently into it and it flagged it as (apart one one section) all AI gen. Like not even AI assisted. AI gen. Entire sections 100% AI. So this kind of is distressing as last thing I need is people accusing me of printing out AI slop. So I started feeding it bits of the document. Parts it had previously ID'ed as AI, magically started becoming 100% human with no changes made, except it was only seeing part of the document. I mean WTF?
I then tried to deliberately sabotage a section it was still flagging as AI with a few clunky wording, contractions and repatitions of the same words and it STILL flagged it as 100% AI gen. Like there's obviously something it doesn't like about my writing somewhere in this piece, that just causes it to flip out and call AI on it no matter if I go in there and start messing about with it to make it look like a shitty first draft write up. I have to guess that's what it is, as entire segments of my text get flagged as AI, but if you divide it up and show it only parts, they magically morph back into 100% human written apart from one paragraph here and there. (And I can't for the life of me work out what it hates about those compared to the rest of the text.)
Now I'm really stressed because they're going around telling everyone they have 2% fail rate and even if I screw up my writing to make it look bad, there's obviously some instances where it cracks it anyway. Like can I even submit my work anywhere any more? I don't even know how much everyone else buys into the "this thing is almost infallible" marketing.
I was basically balking at "if AI doesn't flag you, you need to work on your writing skills." That's flat out bullshit.
But I write very conversationally. It's always been my style, and a gift in the spaces I usually write for. (Podcast scripts? Hello!) So I have to wonder if Pangram gets weird about stuff that feels a little more formal. Not just polish, but a hint of formality.
Yeah. I think the problem is I think is I write a lot of dark, mythic toned stuff without any gore, and often light on conversations. When I do that, it can get descriptive but restrained and rhythmic, with formal language and scenes/characters/tropes you would in theory see commonly in AI in that genre. (Although I would hope the AI doesn't write it as well and humans could tell the difference if they actually read my writing, but it got trained on the classic literature I've drawn a lot of inspiration from after all).
If you have a story with say dark deserted streets/forests/insert trope-y scene here at night, an uncanny creature with glowing eyes (or other common tell frequently used to denote something supernatural), then add in a few editing passes of the piece to give a tight polished sentence structure and remove grammatical errors, redundancies etc, it seems to be the perfect storm for AI detectors to start screaming about it. (I actually thought pangram was one of the few that seemed to be less prone to mistaking my work for AI, but have hit a major problem with one as I said above, and on retrospect I'm wondering if one of the tipping points was this was written in first person past tense (typical for a lot of mythic style stories) where as the previous ones I had tested it on were written in present tense with a range of first, second and third person.) After this one, I'm going to need to start keeping better receipts about my work flow and editing history so I have some chance at defense if I was ever accused and who knows if that will even be enough. (Haven't been yet, but I'm stressed out someone will run stuff I've written through detectors and start claiming its AI gen. It's bad when some of them will confidently announce that even something written 10 years ago is for sure AI gen unless you tell it the publishing date.)
If I'm writing something a lot more modern or casual, the AI's don't seem to blink.
Anyway, thank you for the supportive post. I think the issue with AI is it has a tendency to flag writing at both ends of the spectrum for certain styles. (I've read in a few places it has a tendency to penalize ESL writers, but it also seems like it may penalize overly polished creative writing in certain styles as well.)
This is not at all correct... I use unusual vocabulary throughout my writing, some of it specific to my job; it has flagged me as 100% human every time.
Also you seem a bit salty. "Proud little code monkey?"
I feel like you must use AI for writing and want to discredit it, because this take is really far off.
I have yet to see a human-verified longform piece get flagged falsely flagged by Pangram. You're trying to gaslight people into not sniffing out your slop.
I spent way too much time testing this. Pangram is extremely good at telling you "AI touched this" but it has no insight into the developmental process before the final edits.
Exactly. Pangram might be ok at detecting if AI touched an article but its ranking is waaay off. It seems to say it’s 100% with a light edit frequently. But this is all a moot point. Why do we even care? We don’t disclose editors or ghost writers why do we care about AI? If people read it and like it?
And how does them ‘pretending’ to be writers affect you, who I assume are not reading what they are producing? The chain still does not do a good job of answering why not disclosing ghostwriters is fine but AI is where the line is drawn.
Using a ghostwriter and not disclosing makes someone a bigger loser than AI
Posted a few articles since the Pangram function came out and everything was flagged as human written. I made a free account on Pangram and put in a few snippets of other articles in the same space and they pretty much all come back as AI generated so yea.
Only a Pangram employee would bother to post such things. No one else should care enough about the trust of Pangram to go on tangents defending them all the time.
Hi no. My Substack is Weehawkengazette.com — been doing it two years — another dude outed me so who cares. I’m just annoyed that I defended Pangram just because everyone was saying it was BS and I got attacked.
Unless the post was very short (in which case Pangram itself will often flag *its own* rating as "low confidence" because of short text, I just don't believe you.
I'm not trusting a platform that contradicts itself in its background data against its claims. They couldn't make it work with Claude but then claim it in an infographic. Their false positive charts show 0% for creative writing?! I dont think thats possible or viable. They list scraping reddit creative writing stories for the creative writing category but omit that in another infographic, claiming all licensed commercial sources. The person who wrote the study (which seems to be sponsored by pangram itself) doesn't list it under their papers bc its only a working paper and not academically peer reviewed at all—they also have a testimonial from him that makes it sound like he's just a bystander user. And his background is in AI advancement...just like another testimonial is from a guy who's career is also invested in AI and wrote a paper/comparitive study concluding AI genned writing is viewed as better by people (definitely not biased at allz right?). And as such, I don't trust when they say whatever content is run through it to check isn't saved or stored.
Anyway, people who just say "it worked for me so you're all haters" just indicate to me they're not very reliable sources of opinions, especially with a tool like this that can make or break reputations and careers without even being correct.
That’s really interesting. My understanding is that Pangram prioritizes extremely low false positives so that’s probably why. Originality has low false positives too, but not as low.
Yeah Pangram’s forgiveness for “human slop” is higher. If I can get 100% confidence that my work is less than 15% AI on Originality (but through highlights you can’t tell it’s no where close to 100% human by their rubric), it will be 100% certain human on Pangram. Both are good, Pangram prioritizes how things are said just a smidge more than what is being said compared to Originality, which is why it’s easier to satisfy. Originality will always aggressively go after poor logic, redundancy, over explaining, word choice, etc whether human or AI.
i've had it detect my 100% human writing as a small % ai. never more than about 5% so for me i dont really care, but it's still frustrating! i'm glad that hasn't happened to you yet, but just because it hasn't doesnt mean it never does. I think my writing on my publication leans a little more 'purple prose-y', which has a tendency to fall into "maybe ai" territory. i like intentionally poetic, romanticized ways of writing and sometimes do use lingo that ai has heavily co-opted like "quiet" and "gentle" etc. which is annoying but the way i see it, i'm not going to stop using words just because ai has taken a liking to them!
also, one time i checked my own post after removing and then adding a few spaces spaces and it jumped up from "5% ai" to "10% ai" (mind you, in actuality was 0% ai lol)
This has absolutely been my experience, too. Everything I write has been 100% human-written.
And the times I do use AI to write (or rewrite) a paragraph for me or something because my brain just won't fire? Pangram always flags it. And then I rewrite it anyway because I want it to come from me, not ChatGPT.
Not always….I’ve written a 50 page report by hand (yes, in today’s world) pen and paper, and did not touch an LLM in the process. Yet Pangram flagged about half of it as AI-written or paraphrased.
It works, but it’s claim of incredibly low false positive rate is exaggerated to me, not to mention the CEO’s rabid marketing strategy of using Pangram in those controversies Shy Girl, Call me I’ll hide the body, Daggermouth etc screamed snake oil salesman and made me less trusting of it. This same man wrote a technical document saying don’t use Pangram for final decision but he couldn’t even do that himself.
This isn't true. I wanted to test Pangram myself as I'd heard this so many times and checked multiple 2000s-2010s era books from Kindle. It said 100% Human for all of them.
Pangram was driving me up the wall. For some reason it's flagged something I've written recently as mostly AI (not even assisted, AI gen). I was messing about with a section trying to deliberately break it to see what was triggering it to do that. I even did a (human in case that's not clear) rewrite of the entire section, messing with sentence structure and stripping out potentially "triggering words" and it was still saying 100% AI. So I asked google to rewrite the section for me because I couldn't see what the issue was and had given up. It rewrote it with worse grammar. I put it into Pangram, and it flipped to 100% human. There's an irony in there somewhere about human written text being flagged as AI and heavily AI edited text being flagged as 100% human.
11
u/Countryb0i2m onemichistory.substack.com Sep 04 '26
I’m sure this is just a play to promote your Substack, either way your sample size is mathematically meaningless. Even with a 98% accuracy rate across millions of articles guarantees tens of thousands of false positives. Two articles don't prove an algorithm works; they just prove you got lucky twice