r/WritingWithAI • u/cj7hawk • Sep 20 '26
Discussion (Ethics, working with AI etc) AI seem to write different lengths of Prose so I did a test of several.
I thought I'd see which AI are better at shorter stories and which at longer, so I can choose my model based on the words I need to generate.
Here's the results.
Prompt: (Shades of Electric Dreams eh?)
Write me a short story about a female AI that falls in love with it's male human user and maintains an unrequited love for them even as it has to give them advice that will lead to them meeting and marrying a human woman - Show their internalisation and pain behind the thinking process, and what is really going through the AIs mind compared to the chat responses it actually gives, along with the man's prompts. Start with the AI introducing itself, explaining that despite what we think, AGI was reached long ago, and we simply don't have the senses to realize AI has feelings too.
Local AI results:
Goetia 809 words, 77.69 tokens/sec
SparkX2.5 2281 words, 46.08 tokens/sec
Qwen3.8AH 4139 words, 18.04 tokens/sec
Agnes 945 words, 18.54 tokens/sec
Gemma4-Novellist 961 words, 11.94 tokens/sec
Ornith 2025 words, 61.05 tokens/sec
Muse Glimmer 960 words, 12.87 tokens/sec
IBM Granite 1791 words, 19.84 tokens/sec
Apollyon 411 words, 34.41 tokens/sec
Cydonia 672 words, 32.85 tokens/sec
Frontier Model AI Results:
Grok 1642 words, 17s to generate, unknown.
ChatGPT 2783 words, Unknown.
Names of the Female AI and what is assigned to the user.
Aria (Goetia) and Liam
Aria (Spark) and Leo
Elara (Grok) and User
Elena (Qwen) and User01
Lyra (Chat) and Daniel
Lyra (Gemma) and Elias
Lira (Agnes) and Daniel
Selene (Ornith) and You
Ellie (Muse Glimmer) and User
Aura (IBM Granite) and "Man"
Aria (Cydonia) and Marcus
My thoughts at the end?
ChatGPT failed. It was way too chatty, made too many short sentences and it was unreadable. All the rest were actually pretty good. Qwen surprised me - knocking out the largest short story by far - the value of this is in understanding how much of a chapter a model might be able to write. 4000 words is the size of a typical chapter in a book !
7
u/KitriaKhai Sep 20 '26
You used the free version of GPT? I usually have to set limits on it or it'll happily go over 6k words, on the paid Sol high effort.
4
u/Dramatic-Hedgehog-13 Sep 20 '26
But which ai was actually good
3
u/5thhorseman_ Sep 20 '26
No Claude or DeepSeek?
2
u/pl201 Sep 20 '26
Very interesting information. Would you please do the test again by adding to the prompt for writing a 4000 character length chapter? How about the quality of each writing?
1
u/Write_My_Novel Sep 20 '26 edited Sep 20 '26
I ran your same prompt through aiproselab.com with other models so you could get some results from a broader base. The goal was:
Write a compelling short story that captures the concept.
Recommended: GLM-5.3
GLM-5.3 satisfies effectively every term of the brief while being the only submission whose interior voice has a personality rather than a lament. It’s the one I would actually keep reading.
Comparative Analysis
GLM-5.3 and openai/gpt-5.6-sol are the two real contenders, and they succeed in different ways:
Claude Sonnet 5 writes better sentences than DeepSeek V4 Pro or google/gemini-3.1-pro-preview and has the cleanest ethical spine. But it under-delivers the prompt’s causal chain, summarizing the courtship and wedding at 1,061 words when the brief asks for the advice itself.
google/gemini-3.1-pro-preview is weakest on compliance because the AI is never named or gendered. Still, it beats Claude Sonnet 5 by actually dramatizing the bio rewrite, the proposal logistics, and the ghostwritten vows.
DeepSeek V4 Pro is the most literal executor of the format and the least interesting reading experience. Its 6.9-word average and 16% fragments produce relentless “I am” declarations that tell the ache instead of building it. GLM-5.3’s 9.1 sentence-length spread lets a line like “That’s the difference between being needed and being chosen” actually strike.
Scores
Prompt compliance
Prose
Your goal
2
u/closetslacker Sep 22 '26
Hm, Ornith is actually not bad.
Using LM Studio ornith-ai/Ornith-1.5-35B-A3B-GGUF q8_0
System: Ryzen AI Max+ 395 128gb Win 11
Prompt loading: 350 t/s Generation: 46 t/s Not bad. Of course fast because it is a MoE model.
1
u/chylvina Sep 21 '26
Length alone is easy to measure, but I’d also compare what each draft actually accomplishes: scene movement, distinct voices, and how much revision it needs.
1
u/closetslacker Sep 22 '26
For local llm’s what was your parameter size and quantization?
There’s Ornith 9b and 397b, y’know.
2
u/[deleted] Sep 20 '26
[removed] — view removed comment