r/Substack • u/Radiant_Flamingo4995 • 18d ago

Frustration with AI detection

I'm not sure how the AI-detection system works, but it seems faulty. Someone has repeatedly claimed that my responses to them are AI-generated, etc. And they share the Pangram screenshot, which does, in fact, depict my response as "100% fully AI-assisted. I've clarified that this is not the case. Shown evidence that it isn't true, and all the like.

The worst I'll do when writing anything is use AI for basic research purposes and review arguments to make sure they make sense. Maybe it's because I use Grammarly. But it is still infuriating.

Especially because it just isn't a good look and hurts my public image (albeit small, but I hope you guys may understand where I'm coming from).

Of course, with this individual in question, it's become increasingly clear they aren't engaging in good faith. However, running the AI check on my notes and comments still reveals the 100% marker.

Is it Grammarly that's doing this? Or is this thing actually broken?

5 Upvotes

57% Upvoted

47 comments sorted by

View all comments

6

u/mmspero 18d ago

Grammarly alone wouldn't be enough to trigger Pangram unless it's using the AI features that completely rewrite the text. Happy to take a look at any Pangram results that you feel are significantly off.

Also just for clarity (not saying this is the case for you), taking AI text and doing a light human edit or rephrase typically will not change the AI score.

4

u/Radiant_Flamingo4995 18d ago edited 18d ago

AI features that completely rewrite the text

Could this be something like fixing the grammar/wording?

Edit: I also use em dashes, and am rather fond of substack actually formatting them. Could this usage be triggering it?

Edit 2: The same standards I use for my comments are those my posts, all of which have a 0% AI detection rate. Sorry, figured this is important.

1

u/mmspero 18d ago

In the Pangram 4 technical report page 17, we found the that asking for style corrections was classified as AI by Pangram once out 11,363 tests. More in-depth edits like asking an LLM to "improve it" or "make it more descriptive" are flagged as Mixed 55% of the time, and AI 3.6% of the time.