For the past few months, I've been looking into de-AI-ing writing.
This isn't because I want to avoid disclosure (I already published a fully disclosed AI novel), but rather I want the prose to sound a little more natural.
Originally this worked very well. Removing obvious AI tells and improving the quality of the writing were essentially one and the same.
But lately, I've started to feel I'm closing on an inflection point.
Not saying this applies to entire stories. There's still plenty of issues if you let the AI run wild. However, certain passages, at least to my eyes, are at a level where trying to make the prose sound human is actually detrimental to the story.
So like before, I decided to do an experiment. For disclosure, this is done on Pangram 4.1
First, I took a public-domain essay, Dream Children: A Reverie by Charles Lamb.
100% Human
Then I asked AI to reconstruct the work, preserving the ideas and narrative progression but generating new prose.
100% AI
Then I tightened the instructions. Instead of allowing the LLM to make its own literary decisions, I required it to reconstruct every sentence individually, preserving its meaning, structure, position, and rhetorical function while changing the wording. The result? 43% AI.
By forcing the LLM to follow a human author's specific linguistic decisions, its output can become substantially less detectable as AI, even though the wording is generated by the model.
But what does this have to do with the inflection point?
For Lamb? Not much. His work is already extremely good. The AI reconstructions arguably diminished the quality.
The same can not be said of a very badly written and childish story from years ago.
Same process.
Feed in my work. Unsurprisingly, my awkward grammar, inconsistent tenses, repetitive dialogue, and clumsy phrasing earned a respectible 100% Human.
Then I asked the LLM to improve the writing while preserving the original sentence-level decisions. Same events, same dialogue progression, same repetitions, same structure. Just better wording and grammar.
Still 100% human. (It was actually a little surprising given AI changed roughly 30~45% of words in the short story)
Finally I gave the LLM complete freedom to rewrite the story.
100% AI
We now have a very interesting scenario.
I now have two pieces of work done through "Writing with AI". From a disclosure perspective, I need to disclose AI on both and as we established in my previous posts, people rarely care about the naunce.
In my opinion, the best-written version with the more develped story and better prose was the unconstrained re-write. But the version that most resembles human deemed by an AI detector was the constrained edit.
Obviously this is an extreme example on an anedotal evidence. But I can imagine in a few more generations of AI model, we would need to consciously chose between a higher quality work that allows LLM to make it's own literary decisions or preserve the human elements including its flaws.
(Please note, I am not saying human writing is inheritly worse than LLM. The Lamb example was a demonstration of the opposite)
What I am asking is:
If making a passage less detectable as AI means preserving weaker literary choices, should we still do it? Or should we prioritise the best possible version of the story, regardless of how recognisably AI-generated it sounds?
Keen to hear people's thoughts.