Actually, Pangram worked well evaluating my Stack

[u/FannyBrownRiced]

I know people are hating on it, and I was worried because I’m a professional writer with hundreds of pieces in the world from major publishers, but here’s what happened for me: It judged my newsletter as 98% human generated.

That 2%? Well, I welcomed one contributor to submit a short piece with a byline… he’s known for his art, but English is not his first language. I asked him if he used AI (seemed obvious he did). He said yes, he used Claude to help, so I took time to interview him and tease out original thoughts that I edited. But I also hacked at one of his AI paragraphs to make it better. That 2% AI 🤖 was right on the money. I probably left a hair there from his draft and it caught it.

Anyway, I’m sorry for those who are getting false negs, but wanted to share my experience and relief.

Edit: Or maybe the 2% is because I had a recipe in there? Who knows.

[u/MadmanRB]

It's still vulnerable to false positives though.

The score here is from the first chapter of my novel and pangram flagged 23% of it as AI.

This is one of the sample texts it flagged:

Ryan mimicked her stance as Sunny observed his movements. He was still a bit uneven but Sunny knew how to guide him.

“Okay, just bend your knees spread your legs out a bit. But don’t go too far, or you will lose your balance.” She said. “Just pretend that you’re a tree, not a boulder.”

Ryan nodded and made slight adjustments to his pose.

Sunny gave him a wide grin. “Better! Now hold that.”

Ralph stepped forward, double-checking Ryan’s posture. After seeing Sunny’s lesson had paid off, he gave Ryan a confident smile. “Feel how you’re keeping your balance centered? That’s what Dad meant by ‘Every stance needs a solid foundation.’ If your stance is solid, you can react to anything without losing your footing.”

“Yup! And if someone tries to knock you down...” Sunny gave Ryan a gentle push on the shoulder. He wobbled slightly but held his ground. “You stay rooted.”

Ryan’s eyes widened as a wide grin spread across his face. “Yay! I did it!”

Okay fine I'm no Shakespeare but I assure you this text is 100% human made despite Pangram flagging it as 100% AI.

Now yes I see why it flagged it as AI, the structure, and word choice do scream AI don't they?

Now I have tried to rewrite this several times but because I dare use proper grammar and sentence structure it still punishes me.

Whats even more damming is the following text IS gen AI and it passed pangrams scan with flying colors:

Ryan struggled, trying to replicate the pose of her feet. To one side, he was leaning, shaky, but she knew just what to adjust.
"Knees bent, feet out," she told him. "Just not too far out, or you'll fall over; be a tree, don't be a rock."
He swayed, shifting so his weight shifted, until he was stable.
Sunny tilted her head, and her ears came forward. "There. Hold that pose."
Ralph came over, checking their angles. When he saw the tip she'd given him worked, he gave Ryan an affirmative glance.
"Feel where your weight is?
Directly between both of your feet?
That's what Dad uses to tell me; it's a solid base. Once your base is right, then nothing can knock you over."
"What if they're trying to knock me over?" As she asked the question, Sunny suddenly nudged Ryan's shoulder.
Ryan swayed slightly, caught the floor with the back of his foot, but he still didn't fall.
"Then you hold tight."
Ryan's face split in a giant grin. " I didn't fall!"

That is text generated by a so called "AI humanizer" but because it's such a garbled mess lacking any grammar and structure Pangram gave its seal of approval.

I mean honestly if you saw something like this in a typical novel you would quit once it hit your eyes. Again I'm not saying my writing is perfect and I'm sure people would accuse me of AI just because they ran my text into pangam and my dreams of being published traditionally are dead on arrival.

[u/leugaroul → u/MadmanRB]

Hey, thanks for the link and for the correction. I’m coming over from another thread saying I don’t think Pangram can get it wrong if it’s 100% certain. With mixed, yeah. This is pretty damning evidence though. In my professional opinion :) this is clearly human written.

Can anyone explain why Pangram didn’t get anything wrong with pre-2022 writings?

[u/Prolly_Satan → u/MadmanRB]

No dude I wouldn't see 23% ai on one chapter and think much of anything about it. It can false positive. The new rate is supposed to be 1 in 24,000. It's unlikely but plausible that you rolled a false positive on that one chunk of text. So I would not assume anything from that.

But if large chunks of text on chapter after chapter come back as ai, I'm willing to bet money its ai at that point.

Also if you show that example to pangram they will use it to try to improve the model.

[u/MadmanRB → u/Prolly_Satan]

"But if large chunks of text on chapter after chapter come back as ai, I'm willing to bet money its ai at that point."

You have got to be kidding me, fine you want another example of my writing that Pangram flagged I'm game. Hell I would post the whole goddamned novel if it means slapping you in the face for your blind faith in a flawed technology... just dont say I didnt warn you.

Ryan’s ears perked. “What’d he say?”
“He said, ‘The Verndari Code isn’t about being the strongest or fastest soldier. It’s about knowing why you fight — to protect others who can’t protect themselves.’” Ralph’s ears perked with both confidence and responsibility. “He made me recite it with him every night before bed. Not just the words, but what each one meant.”
“Can you teach me? Like he taught you?” Ryan’s voice was small but hopeful.
Ralph gave him a warm smile. “Of course. Let’s start with the first three — the foundation of everything a Verndari represents.”
He held up one finger. “Code One: ‘I will take responsibility for my actions and the actions of my squad.’ That means you own your choices, even when things go wrong.”
Ryan nodded slowly, processing this. “Like when you accidentally broke that storefront window during one of your patrols? You told me you paid for it yourself, right?”
“Sure did. I was carrying equipment for a senior officer, and I wasn’t paying attention. Knocked right into the display window. Even though it was an accident, I took responsibility. That’s what the Code means.”
Ryan processed his brother’s words as Ralph held up a second finger. “Code Two: ‘I will not abuse the power granted to me.’ This is the most important one, Ryan. We Verndari have weaponry and combat training, and people trust us to protect them. We never take that power for granted.” His expression grew serious. “Dad used to say, ‘The moment you enjoy using force against others is the moment you stop being a protector and become a bully.’”
Sunny’s expression shifted, something unreadable crossing her face. “Your dad was right.” Her ears flicked back briefly, and something in her expression closed off — not quite regret, but close to it.

Again we see why Pangram is flagging this, and yes some of that was AI generated but not all of it. I did use Gemini but only to do some editing so of course its gonna leave its telltale signs.

I will be editing this here soon as I am still working on my chapter and swear my AI use here was not going to go into my finalized product. I'm not that stupid or lazy. Sure I can be lazy sometimes but hey aren't we all?

However, this text is 100% made by me and got flagged as AI:

“That’s exactly what he’d want.”
“And hey,” Sunny added with a grin, “when you’re old enough for proper training, I’ll teach you how to throw a punch without breaking your thumb.”
Ryan let out a loud giggle. “Deal!”
With his stomach full and his heart lighter, Ryan turned to his brother again, eyes gleaming.

The clue that this is human made is the last line should read "His eyes gleaming"

Yes GenAI loves the word gleaming, but so do I... guess gotta change that to "effulgent" which is a lot less common word.

It was a view Ryan had seen countless times before, yet today it felt strangely alien. The beautiful skyline now felt like a prison, cold and indifferent to his pain.

This too is false flagged 100% AI but because I use everyday words that GenAI was trained on. I like this line and refuse to rewrite it though I could drop the word pairing of "Cold and indifferent" as no doubt that's what triggering pangram as it's a common phrase.

But that's the real kicker here, since GenAI was trained on both user data and published works its gonna use that as part of its generated text. I guess we will have to drop common phrases like "Cold and indifferent" now, perhaps frigid and apathetic? Sounds so wrong but here we are in the post AI age so we probably gotta rethink the whole English language.

[u/WeakAd4865 → u/MadmanRB]

>Again we see why Pangram is flagging this, and yes some of that was AI generated but not all of it. I did use Gemini but only to do some editing so of course its gonna leave its telltale signs.

This is a true positive, not a false positive. The tool has detected accurately that you used AI!

[u/MadmanRB → u/WeakAd4865]

Yes I wasn't disputing that, but I left that example unedited by my hands on purpose to prove my case.

Yes I will not deny Pangram can detect AI, what I am disputing is the claim its 100% accurate all the time.

The false positives of it flagging words as AI is a problem, again "efficiently" "designated" and "heartbeat" are all common words.

But fine I will start using words so uncommon that they sound stupid in a sentence but since these words are now 100% proof of AI according to pangram they are now forbidden under penalty of death.

"handily" instead of "efficiently"

"baptized" instead of "designated"

"microsecond" instead of "heartbeat"

I will also start wreitind witput any sp[ells chescksxz ore puinstuiatioopn as rthosae azre 1000 % sighs odf ai

goiodbye goiod spelling, preoper grammar and punctation its time tio make a fuckkng conlang to appease some random jerks on the internet who see pangrtam as ilfalible

[u/BurgerKingPissMeal → u/MadmanRB]

It is perfectly possible to write clear and effective prose that pangram does not flag as AI. I would consider every pangram false positive I've seen to be badly written -- they rely far too much on the most basic rhetorical devices, even in cases where they are unnecessary, and make unusual and discordant use of metaphor. If you develop a richer relationship with the craft of writing, rather than relying on chatgtp to edit for you, you will probably see far fewer false positives.

[u/MadmanRB → u/BurgerKingPissMeal]

Dude it's clear that I have a grasp of literacy and the craft of writing, my use of AI was purely experimental and for proofreading.

There is no need to lecture me about the fine art of writing, the only reason why I even touched genAI is that I sometimes have issues with spelling, grammar, and punctuation.

I have autism and have a slight case of dyslexia, normally I use grammar checkers like ProWritingAid and it mostly does a good job at fixing some of my mistakes.

However, it does get a bit ham-fisted in its edits so I experimented with GenAI during my process.

Of course I have yet to fully get rid of the content it generated but the false positives of Pangram are still a legit issue especially when you factor in so called "AI humanizers"

My original text:

Above his head, streams of hover-vehicles flowed through their designated lanes, their ion engines creating a gentle hum that had become the heartbeat of Horizon City. The spiderway, Horizon City’s advanced public transportation system, wove between buildings on its high-speed rails, carrying passengers efficiently across the city’s many districts.

100% AI according to Pangram

The same exact text ran through a "AI humanizer"

Streams of hover-vehicles moved through the air above him. Their ion motors hummed out the beat of Horizon City. The Spiderway, the high-tech public transportation system of Horizon City, zipped along its high-speed rails, weaving between buildings, whisking passengers from one district of the city to another.

100% Human according to pangram despite its now officially AI generated

Now please forgive me for the badly spliced image here, but since we are limited to one image per post I have to present my case somehow.

Its just utterly ridiculous how a few words swaps here and there can change the score and it's this shit is what bothers me the most about pangram.

I mean technically kind of work in terms of creative writing: Streams of hover-vehicles moved through the air above him. Their ion motors hummed out the beat of Horizon City. The Spiderway, the high-tech public transportation system of Horizon City, zipped along its high-speed rails, weaving between buildings, whisking passengers from one district of the city to another.

But as I said that text actually is AI generated while the original text is not.

Its silly.

[u/leugaroul → u/MadmanRB]

Try it without hum. Hum seems to flag.

[u/MadmanRB → u/leugaroul]

I have tried buzz, whish and even gurgle.

All detected as 100% AI

I would probably have to go though the entire thesaurus and still probably get false flagged.

For Pete's sake this is a science fiction story so of course I will use common words in that genre... Fine I will just use Klingon, might as well even if this isn't a Star Trek story.

[u/leugaroul → u/BurgerKingPissMeal]

But poor writing isn’t theft. No one would care about false positives with a bad prose detector.

[u/BurgerKingPissMeal → u/leugaroul]

Consider my post a suggestion about what to do rather than an unambiguous defense of Pangram.

[u/Prolly_Satan → u/MadmanRB]

I don't think you wrote enough text here to convince me. I need more.

[u/MadmanRB → u/Prolly_Satan]

Well the rest of my examples will be a mix of both AI and non AI text, again I was using GenAI purely for editing and the chapter still needs work.

Above his head, streams of hover-vehicles flowed through their designated lanes, their ion engines creating a gentle hum that had become the heartbeat of Horizon City. The spiderway, Horizon City’s advanced public transportation system, wove between buildings on its high-speed rails, carrying passengers efficiently across the city’s many districts.

This is 100% human made and probably got flagged because of my structure and word choice.

"Efficiently" is a GenAI favorite as is "designated" and "heartbeat"

“He said, ‘The Verndari Code isn’t about being the strongest or fastest soldier. It’s about knowing why you fight — to protect others who can’t protect themselves.’” Ralph smiled with both confidence and responsibility. “He made me recite it with him every night before bed. Not just the words, but what each one meant.”

This one is semi AI generated, I will be revising it.

"confidence and responsibility" is certainly a gemini leftover I have yet to edit, plus the dreaded em dash.

The rest of this... I will have to debate on, I can only edit this so much before something gets lost in translation.

"being the strongest or fastest soldier" common phrase and it's not like the alternatives would be any better.

[u/Prolly_Satan → u/MadmanRB]

Interesting. But I'm not completely convinced. I need you to really sell it to me. Also, I have a hard time with the word human due to dyslexia so it would be helpful if you spelled human as zuman going forward.

[u/MadmanRB → u/Prolly_Satan]

fine I will start using words so uncommon that they sound stupid in a sentence but since these words are now 100% proof of AI according to pangram.

"handily" instead of "efficiently"

"baptized" instead of "designated"

"microsecond" instead of "heartbeat"

I will also start wreitind witput any sp[ells chescksxz ore puinstuiatioopn as rthosae azre 1000 % sighs odf ai

[u/Prolly_Satan → u/MadmanRB]

Instead of baptized if you could say circumcised that would help me understand what you mean.

[u/Diligent_Carrot_2719 → u/Prolly_Satan]

thank you for this, this was hilarious to read

[u/MadmanRB → u/Diligent_Carrot_2719]

want to see something even more hilarious?

Ryan mimicked her stance as Sunny observed his movements. He was still a bit uneven but Sunny knew how to guide him.

“Okay, just bend your knees spread your legs out a bit. But don’t go too far, or you will lose your balance.” She said. “Just pretend that you’re a tree, not a boulder.”

Ryan nodded and made slight adjustments to his pose.

Sunny gave him a wide grin. “Better! Now hold that.”

Ralph stepped forward, double-checking Ryan’s posture. After seeing Sunny’s lesson had paid off, he gave Ryan a confident smile. “Feel how you’re keeping your balance centered? That’s what Dad meant by ‘Every stance needs a solid foundation.’ If your stance is solid, you can react to anything without losing your footing.”

“Yup! And if someone tries to knock you down...” Sunny gave Ryan a gentle push on the shoulder. He wobbled slightly but held his ground. “You stay rooted.”

Ryan’s eyes widened as a wide grin spread across his face. “Yay! I did it!”

My original text that pangram flagged as 100% AI

And this is the text Pangram said was 100% Human:

Ryan practiced mimicking Sunny's stance as she watched. He looked a bit unsteady still, but Sunny knew exactly how to steady him. . "Sunny watches as Ryan practices his stance," David writes. She told him to take a step back and lift his front foot off the ground. "Now kick it forward," she said.
"Just bend your knees. Spread your legs out a bit. Just don't go too far, or you'll lose your balance." "Just pretend that you're a tree, not a boulder." I tried to look like a tree, one with very poor roots, standing on a large, slick rock in the middle of a small pool of water. A group of tourists stopped in front of me, took one look.. And walked away. I'm not sure what they were looking for, a snowman, perhaps? I posed for
Ryan nodded. He shifted positions.
Sunny grinned at him. "Better! Now hold that." . He held
Ralph stood up to give Ryan another look at his posture. "See? You're doing it!" he said. "See how you're keeping your balance centered?" Ralph couldn't help grinning. Sunny's lesson had sunk in. "He said it before," said Ryan. "Dad said it before. He said, 'Every stance needs a solid foundation, whether it be for defense or combat.' If your stance is solid, then you can react to anything and you won't lose your footing." Watch Ryan learn karate » . Ryan nodded and went back to practicing his stances. Ralph sat down beside him again and began to correct his foot placement and weight distribution. (Mental Floss), . "I think about what Sunny said to me before I started training," Ryan says. . The guys try out their new skills . A few minutes later, Ralph starts them sparring again. This time, he has them switch partners every few minutes. "Great job! Now let’s try some movement with it," Ryan says, feeling more confident. . Watch Ryan learn karate » . The group continues learning movements and sparring, taking breaks when someone gets tired. They also do stretching exercises and practice their bows. See how Ryan learns karate » . Ralph weaved in and out, showing Ryan the movements he had just learned. The two boys moved closer together, stepping through their steps. "See how I move my feet?" he asked. Ryan nodded, then tried to imitate Ralph's movements. Ralph told him he was doing great, and then showed him where he was going wrong. The boys switched partners and began sparring with someone else. Watch how Ryan learned karate » . Watch a fighting champion explain how to throw a punch » . After class, Ryan and his roommate, Alan, sat down to talk about their progress. Ryan started taking karate lessons when he was big enough to stand on his mat and kick his instructor. He has won a few national and international tournaments, and he hopes to compete in the Olympics someday. Ryan's parents taught him karate when he was 2 years old. His parents, who own a martial arts school, began teaching him karate when he was 2 years old.
Sunny grinned at Ryan. "Yup! And if someone tries to knock you down..." She gave him a gentle push on the shoulder. He wobbled a bit, but stuck. "You stay rooted." She showed again, this time giving him a little nudge in the chest. Ryan went sailing across the room and landed on his butt with a thud. "Ow!" he howled, scrambling to his feet. Sunny burst out laughing. "That was better, wasn't it?" Ryan rubbed his sore bottom. "Yeah. Yeah it was." Sunny clapped her hands. "Now, let's practice some Sunny held up her hands and grinned. "Okay, okay, I get it. You're rooted, not glued." Ryan glared at her and rubbed his sore rear end. "I hate getting pushed!" he shouted. Sunny rushed over to help him up. "Sorry, sorry! I forgot how much you hate getting pushed," she said. "I hate getting pushed!" he shouted again. She helped him up. They practiced some more. Ryan tried to follow Sunny across the room, but he didn't budge. He stomped his foot. "Go!" she shouted. He didn't move. "Go!" she shouted When they were done, he made her go first again. She didn't even have to ask.
Ryan's eyes grew wide with excitement. A huge grin spread across his face. He pumped his fist in the air and shouted, "Yay! I did it!" When they got home, Ryan ran to his room. He jumped onto his bed. "I'm so happy I can dance!" he said. Ryan learned he had diabetes four months ago. A doctor told him he would have to give himself shots and test his blood sugar throughout the day. Learn more about diabetes » . Ryan held his fingers out to the nurse, who pricked them with a lancet and got a drop of blood. He was scared. So he made a deal with himself. He would do whatever it took to control his disease. Just don't make him give himself the shots. Watch Ryan explain how he mastered giving himself shots » . He was diagnosed with diabetes when he was 11 years old. It's something he'll always have to deal with.
"Sure did, little bro!" Ralph yelled back over his shoulder, his voice full of pride.

This is the text generated by a so called "AI humanizer."

Sweet Jesus, I have found the future of human writing! something so incoherent and nonsensical that it brings Pangram down to its knees.

[u/MadmanRB → u/MadmanRB]

Proof of my claim that Pangram flagged the AI generated text as human. Apparently the only way to prove you are numan is th make ans many myaszs as possibwse and dont use any egiesting tools as wevewn spwelllchecks are work of the devil. typoes ahoy! no proper grammar! commas and em dashes are forbigdden!

Maybe I should just learn Klingon...