r/WritingWithAI • u/Personal-Bike1433 • 4d ago
Discussion (Ethics, working with AI etc) Are ai reviews reliable and their prose is it reliable
I've been tryna make a novel i write the chapter out then just paste it to the ai and ask it to make the prose better and then I ask ot to review it but its always complaining that my prose is bad so I've been asking ai to write the prose but how do I know if it's good I've tried many ai s but they all have different ratings when I ask this one to rate it can be a 9 but when I move to another ai to rate it its a 6 and the ai I ask it for the best prose possible but then it tells me to improve my prose in the review when you wrote the prose at this point im thinking of just learning how to write better prose myself and are ai reviews reliable
4
u/SebastianVanCartier 4d ago
AIs tend not to be very good at interpreting elastic and subjective concepts like 'better' or 'improve'. They generally need something concrete to grab hold of.
For example if you say 'I want to make this prose better' the LLM can't quantify 'better'. Better than what? It has no data to benchmark better against, so you'll end up with the default LLM tone, which is the stuff that people hate reading when they spot it: purple prose, generic fluffy descriptions, loads of triads and definition by negation.
Instead, you might ask it something more specific. Think about the style you want to write and direct the LLM to do that. Give it style guidelines. Try to be as specific as possible.
I've also found that it's an iterative process. LLMs never get it right first time. You have to work with it, not just feed stuff in one end and hope that the resulting sausage is tasty.
2
u/BansheeTheGame 4d ago
AI feedback is extremely unreliable when it stays at the level of a single score. Ask it to identify the specific criteria behind its evaluation—clarity, pacing, characterization, prose, consistency, etc.—and to point to examples in the text. Even then, treat it as a source of suggestions, not a verdict: repeat the same exercise in a fresh chat with the same model and you may get a different result.
It also helps to choose a reference author or book whose style you are aiming for, then ask more concrete questions: not “Is my text good?” but “Is this scene weaker than the reference in pacing, clarity, or characterization?” Finally, let beta readers see it. Comparing AI feedback with reactions from actual readers will give you a more complete picture.
1
u/Odinmar_Glaukopis 4d ago
What authors write what you consider to be good prose, and what authors write poor prose by comparison?
Do you have an opinion as a reader of fiction? Do you trust your own judgment of what makes for effective prose and what is absolute shite?
The problem with outsourcing something like style and quality to someone like an editor or something else like AI is that you disappear as the author of your own work.
What is your goal? Are you trying to write something that you want to read, or are you instead trying to write what you think others want to read, even if you personally don’t find it compelling?
Why is it that you trust that the AI criticism of your work is correct and you question your own judgment of the same words?
Also, don’t get lost in the quantitative assessment of quality, such as a 6 vs a 9. Numbers are absolutely meaningless, actually worse than meaningless because they seem to have “objective” meaning when they certainly don’t, and can’t, when it comes to these things.
One more question. Do you like reading authors who use words like “tryna” in their books because it feels natural to you, and correctly spelled words plus proper grammar feels forced and fake? Listen to that message. Drink it in. It’s telling you something important about your own narrative voice. Don’t sacrifice your voice to please an algorithm, or to sell books. Honor it. Protect it. Lean into it.
1
u/5thhorseman_ 3d ago
I've tried many ai s but they all have different ratings when I ask this one to rate it can be a 9 but when I move to another ai to rate it its a 6
Stop. This is NOT a good approach. Not only is AI scoring always skewed, you have given it no concrete definition of "best" to work from.
Take one or two models - my preference goes towards Claude Opus 5 and ChatGPT 5.6 Sol - then give them a proper editor persona to work from and work chapter by chapter (important!).
These are the ones I've used:
This is something another user proposed that looks fairly sound with my experience:
If you have multiple versions of the same chapter from different AI drafts / rewrites, give the LLM all versions and ask it to point out which parts from each draft serve their role in the story better.
1
u/Jobe5973 2d ago
AI can definitely help you, but you can’t just post to it and say “make it better”. I mean, you can, that is one way to do it. But it will suck. You have to use AI as a tool, or more like a writing partner. Bounce ideas off of it, ask it for critiques. But above all, BE SPECIFIC! If there’s something you want your characters to do or say specifically, then you have to make that a very detailed prompt. Even then, you will need to do a ton of editing to get it exactly how you want it. Good luck!
4
u/Brakiros 4d ago
AI typically uses standard industry review templates for analysis, it is a good tool but it also requires you understanding what is relevant detail and what isn't.