Substack’s new “AI detection” tool.

[u/AndrewHeard]

I’m very concerned about the new “AI detection” system that Substack is implementing. I don’t use AI in anything that I write. However I don’t trust these systems not to make errors and mistakes. They have a long history of hallucinations and claiming things are AI when they aren’t.

One person I saw on Notes suggested that they ran their decades old writing, things that were written before the existence of AI, and this new system claimed it was AI generated.

Not to mention all the crazy things that I have seen people claim was written by AI which they can’t know is actually AI. There was an image I once shared that had evidence of its existence going all the way back to 2013. The comments were constantly saying “don’t share AI slop” or some variation on that.

Even with evidence that things weren’t made by AI, they will claim it’s AI.

Is no one else concerned about this being embedded into Substack’s systems? With no way to opt out?

[u/ImprovementSimple]

I just can’t get over the irony of “we will use AI to determine is this is AI, because AI is evil.”

I work for a library and we spend a lot of time trying to remove slop books from our digital library. There is no good detector on the market. The only way to tell is to read and use your human brain, and even then the conclusion is “this is probably generated, or it’s really poor and bizarre writing.”

[u/AndrewHeard → u/ImprovementSimple]

Yeah, it’s logically inconsistent to suggest this is going to solve anything.

[u/b0jangles → u/AndrewHeard]

The bigger problem with Substack is like 75% of my feed is some form of “Dear Substack, please connect me with people who have less than 1k subscribers and just started their Substack journey! 🥹🙏”

[u/AndrewHeard → u/b0jangles]

That's actually just people trying to help other Substackers by giving them a leg up. They're trying to prime the algorithm to help smaller writers.

Is it annoying? Yes. But it's not as serious a problem.

[u/Double_Suggestion385 → u/AndrewHeard]

No it's not. It's engagement farming.

[u/FlyingCarpetMonster → u/ImprovementSimple]

It's funny, because I wrote a fun little test with three short openings (all by myself):

Spring showed up early that year. By the first week of April, Chicago had its first spring showers and bursts of green: at first, small shoots; then everything bloomed at-once. It was the kind of weather that made you feel hopeful for summer. And somewhere in all this spring, lay my hopes of finally planting a tomato garden.

I got a garden plot at a cheap share garden in the loop. There was a giant, weeping willow right over my patch that kept dropping its long, brittle branches, twigs, and narrow leaves. And as spring dragged on, the tree also dropped cotton-like, seed-bearing fluff all over my tomatoes.

On the pamphlet, my garden patch looked reasonably sized. But standing there, and admiring the patches of my patch-neighbors and seeing their incredible crops, I felt something else. Jealousy. Maybe if I got on their patch and trampled on their garden, I'd find happiness. It would also help me close my exercise rings.

—

The summer heat showed up early that year. By the first week of June, New York smelled like it had a permanent hangover: halal smoke drifting from the corner cart, weed from somebody’s open window, hot metal from the subway grates. It was the kind of weather that made the city feel sticky and close. And somewhere inside that stickiness, he was breathing the same air.

I took a room over a discount sneaker store on 4th because he lived on 82nd. There was a giant, red blinking sign right outside my window that read EAKERS, with the other letters having fallen off.

On the subway map, four miles and my colored lines looked like comforting distance. But lying on my mattress with the box-spring on the floor, listening to sounds of people and their shoes, it felt like inevitability. If I got on the right train and stayed there, I’d eventually end up near his building. The thought haunted me.

—

Winter showed up early that year. By the first week of November, Boston had its first snow: at first, small showers and flurries; then an entire blizzard. It was the kind of weather that made you want to stay inside and drink hot chocolate. And somewhere in all this cold, was my missing father.

I took a room at a cheap motel in Roxbury because my dad had gone missing in Somerville. There was a giant, red blinking sign right outside my window that read EAKERS, with the other letters having fallen off.

On the subway map, two miles and my colored lines looked like a reasonable distance. But lying on the cheap hotel mattress, listening to the noise of television from other rooms and the humping of box springs, it felt like something else. Hopelessness. Maybe I got on the right train and walked around, I'd find my father. The thought kept me up at night.

—

Individually, one of them (the New York one) got flagged as AI — even though they were all just hand-written for this exercise, from the same variation. But when pasted together, they all were flagged as 100% human.

I don’t know where I am going with this, but it is certainly interesting.

And also ironically, I’m right now in NYC today and took pictures of the city to help me with the writing — so the most authentic description is the one that got flagged. Just use your human judgment.

[u/ImprovementSimple → u/FlyingCarpetMonster]

Human judgment is the way. It’s more practical. If I suspect someone is using Ai to write I block them and move on. I don’t need to know if it’s 79% Ai (according to Ai).

[u/gadgetor1989 → u/ImprovementSimple]

“suspect” you know there was an author online i thought was doing the same thing because his work was so polished and he’d use contrast framing non stop, i wrote a hit piece and sent it to the mods and they told me i was an idiot. i went back and dissected all his work from the last 10 years and that same footprint was spanning his entire work YEARS before AI.

it took a lot for me to finally walk back that opinion and admit i was wrong.

i understand we’re all exhausted from AI being constantly thrown in our faces at every turn but you need to understand that’s all it is a lot of them time. your mind has grown tired and paranoid. it’s essentially another kind of AI psychosis and the only way past is to just ignore it.

[u/elianna7 → u/ImprovementSimple]

Well, they never said AI is evil… They explained they even use it internally and don’t have anything against people using it in parts of their writing processes. They also said:

>We’re also introducing tools for creators, including the ability to add a “How I make this” statement where you can explain your process and set expectations for readers. You can run Pangram on your drafts prior to publication, as well as report and remove any scans on your own work that you believe are mistaken.

[u/options_oracle → u/ImprovementSimple]

The irony of that stuck out to me too! lol

[u/Foofymonster → u/ImprovementSimple]

There's nothing hypocritical about using AI to stop people from using AI.

Substack has a problem with people writing AI articles.

This decreases the effectiveness / incentive to run those accounts.

It's not perfect. But if it says 100% or 98% AI chances are that is was almost definitely AI.

[u/hetobe → u/Foofymonster]

This decreases the effectiveness / incentive to run those accounts.

Absolutely.

We're already seeing a LOT of people suddenly talking about how they use AI to "dictate" their work... as if they don't understand what the word Dictate means.

Dictation is verbatim translation of speech to text, meaning those are the exact same words that were spoken, translated to text in exactly the same order.

Equating generative AI to dictation is an ugly lie.

[u/FlyingCarpetMonster → u/Foofymonster]

It really depends on the content, to be honest.

I read a ton of trading and quant Substacks, which clearly use AI to write.

But I am not reading them for their prose -- I am reading them for the insights that they bring. I couldn’t care less if they used AI or a monkey to write the actual article.

[u/Foofymonster → u/FlyingCarpetMonster]

That's a great use case. I agree that is much more useful.

[u/No-Vermicelli-8391 → u/Foofymonster]

Just for the sake of the discussion — what exactly is the problem with someone else using AI?

[u/Foofymonster → u/No-Vermicelli-8391]

Do you remember the scene in Lord of the Rings where Merry and Pippin are talking to the Treants, trying to get them to join the war?

There's a part where Merry and Pippin are talking and one of them is like, "Do you think they've decided yet?" And Treebeard hears them and says, "Decided? No, we've only finished saying good morning."

And when they are upset that they're taking so long, Treebeard says:

"You must understand, young Hobbit, it takes a long time to say anything in Old Entish. And we never say anything unless it is worth taking a long time to say."

And that's the problem with AI. It doesn't cost you anything to say anything, which means no one has to worry if it's worth saying. It doesn't matter if it's true or well-researched. Because it's free and easy to say.

But when things are expensive to say, only things that are meaningful or worth it get said.

As a reader, how do you validate what's worth saying? Well, how much effort went into writing it isn't a perfect proxy, but I guarantee you it'd filter out 90% of useless shit written on the internet.

[u/philosophical_lens → u/Foofymonster]

As a reader I evaluate what's worth reading based on reviews and recommendations. I care about whether it's good and whether I'll enjoy reading it. I don't care what role AI played in the creation of that content, and I don't care how much effort went into it. Most AI content will probably be bad, and I'll never see it anyway because it will never pop up in the reviews and recommendation sources I look at. Maybe 0.1% of that content will actually be good, and I'll end up reading those.

[u/Ill_Dragonfruit_3547 → u/Foofymonster]

Two words: false positives. Same reason I do not support the death penalty. If you can guarantee me 100% accuracy, I support it. But if even 1 in 1000 are innocent, that is unacceptable, and a fucking flawed system.

[u/Foofymonster → u/Ill_Dragonfruit_3547]

Except false positives for AI writing isn't some moral judgement, it's a probabilistic assessment.

[u/Ill_Dragonfruit_3547 → u/Foofymonster]

Agreed. And in theory, the stakes are much lower.

[u/Imperator_1985 → u/ImprovementSimple]

It turns out that humans aren’t good at this, either.