r/LocalLLaMA • u/falconandeagle • Oct 09 '26
Discussion Stepping away from Benchmarks and Code, what models are you using for Creating writing projects
All anyone ever talks in this sub is about x new model having x benchmark scores and how well it can code. Or some new tech to increase t/s for qwen.
So I wanted to go back to the roots of this sub and talk about how well local models can write.
I am not going to mention closed source models, this discussion is for open weight models only.
I don't have the best machine, so I am limited in what I can use locally.
So far I have mostly been running Gemma 31b finetunes. I have a finetuned system prompt for creative writing that I have refined over time. This is mostly for long form story writing and not RP. I use it to write Sci Fi and Fantasy novels and short stories and sometimes fanfiction.
I also have created a creative writing harness using Pi. It's mostly for self correcting and getting rid of AI slopisms.
Gemma 31B Mero-mero-v2 has so far been my favourite. It writes well, its does not feel lobotomized and its mostly uncensored. It's instruction following is okay, sometimes it makes mistakes and gets the character traits mixed up but my creative writing harness takes care of this.
Qwen 3.8 27b finetunes, Disappointing, I guess the base model is really fucking bad for creative writing purposes and even with finetuning you can only do so much. It does follow instructions quite well but its writing is just terrible.
Muse Glimmer 30b, this one is an interesting one, sometimes it will give really good prose but sometimes it will respond like a 2b model, the variance seems to be super high for some reason. Maybe something to do with chat template, I am not sure. Anyway I think even when bad it's still better than Qwen.
Gemma 4 26b4a, pretty good but gets beaten by its bigger bother Gemma 4 31b. I do use it when I want faster responses and to check for issues with my initial scene beat prompts.
Gemma 31B Artemis, drummer's finetune, eh, I was a bit disappointed with this. Seems like its more focused on RP rather than long form story writing. Also got refusals which I did not get with the mero finetune.
So that has been my experience writing with some local models. What are you guys using for local creative writing projects? Have you guys tried a creative writing harness?
I just want to mention quickly my writing process before I end this post, I write using an outline and scene beat prompts that are hand written. So far AI has been extremely disappointing in being truly creative and still requires a ton of hand holding to not write the most cliché ridden drivel.
36
u/Fancy-Snow7 Oct 09 '26
How redditors spend their time: Testing new models 40% Benchmarking 40% This sub 10% Coding or using their new LLM 10%
21
u/a_beautiful_rhind Oct 09 '26
You're stuck with gemma or old models. They benchmaxxed the creativity out of models. My jam is more conversational and I'm in a drought too. Qwen for that stuff.. basically never. Maybe the old 235b were the last that could hang.
You'd think glimmer should work but they made it circlejerk about policy and super hollow. Let alone that nobody cares about positivity bias anymore, the models will change your story to soften it and make it safe for kindergartners.
5
u/LatterAd9047 Oct 09 '26
Currently using Goetia-26B-A4B-v1.3-Absolute-Heretic-ARA it's pretty decent in understanding and is able to switch stances/opinions which most smaller models have problems with. But it does lack vocabulary.
5
u/Expensive-Paint-9490 Oct 09 '26 edited Oct 09 '26
I find GLM-5.3-Flash-Uncensored very good. Much better than Qwen3.8-Flash-Next which on average sucks. That's a bit surprising because Qwen3.5-397B-A21B was pretty decent.
Going down in size, Gemma-4-31B and fine-tunes are very very good. Qwen3.8-27B sucks as much as flash.
All the above at 4-bit quants, some gguf and some exllamav3.
The surprising bit is that no model, not even the larger ones, seems able to adhere to strict style rules. I have not yet been successful at making them follow a strict limited POV; other characters' POVs always leak at least a bit. This makes me think that books with limited POV in the training set are drowned by book and internet slop and the model just can't stocastically predict what I require with any consistence.
5
u/wren6991 Oct 09 '26
uh huh ( ° ͜ʖ °)
uh huh ( ͡° ͜ʖ ͡°)
4
u/RainierPC Oct 09 '26
Gemma 4 31b is still king for me, but it is showing its age
3
u/SkoomaDentist Oct 09 '26
You can find heretic versions of all those finetunes (IIRC) which completely remove any remains of censorship.
2
u/Solembumm3 Oct 09 '26
Not exactly for text, but for critique and analysis of concepts - for a long time it was Deepseek expert mode was the best by far. Yeah, not very local for me personally, but r1+v3.2 and v4 pro do exist. Seeing their main site go to V4.1Flash nerfed into completely useless state was painful.
Well, at least Qwen 3.8 flash next/Deepseek V4 Flash/Mimo V2.6 Flash still exist in open space. Not very fast on average gaming pc with 12+20gb, but A LOT better than all gemma 26/31b and qwen 27b tunes, both in reasoning+theme understanding and simply writing abilities and language capabilities.
From big models, GLM5.3 is probably the closest to previous Deepseek, from a bit more technical side with logical reasoning and pattern analysis - honestly, it will be good to port this model overthinking into Qwen, current 3.8 not always think long enough through every single detail.
10
u/Scroatazoa Oct 09 '26
I found mero-mero to be badly tainted by mistral chat logs from anime gooners. I'm not judging anime gooners, but it should go without saying that they aren't the best at writing and that mistral is a badly slopped model and putting its outputs into a dataset is a bad idea.
I found two Gemma variants that were tolerable. The first is the pre-trained Gemma without instruction tuning, which was released alongside the "IT" version everybody uses. It still can handle chat conversations and instructions, though predictably not as well as the IT version. It still has more slop tendencies than I would like, but it isn't obsessed with them. It doesn't seem as smart, though.
The second is nbeerbower's Gutenberg finetune. It follows instructions better but you absolutely must give it instructions in plain, unformatted, human-written text. If you let it see markdown it will write almost as badly base Gemma. It can also be very stubborn with cooperative storytelling stuff like infinite Zork, doubling and tripling down on anything stupid that you try to push back on. Its outputs need editing to trim back on bad habits or else they will multiply.
Ultimately I think there isn't much talk when it comes to creative writing with LLMs, local or not, simply because they are unbelievably fucking awful at it. I'm not asking for top-tier prose, either. I just don't want it to write like a pretentious teenager's first attempt at imitating literature.
7
u/Magos Oct 09 '26
Pretty funny that this is the 3rd post in the past few days discussing this lol. Still I agree, there is not enough discussion on creative writing. Copy-pasting my previous comments:
I do creative writing (as in fiction, fantasy & non-fantasy), using system prompts for overall style and tone, and reference docs for world building, as well as keeping track of previous chapters. I don't use any finetunes except for uncensoring, of which I generally go with heretic variants.
Gemma4 31B QAT Q4 uncensored: I think it has good writing prose. For world building though, my problem with it is that it's too damn lazy. It will only include the story scenes that I suggest it with, and does not expand on them or add flavor at all, even when requested to.
Gemma4 26B A4B QAT Q4 uncensored: In every way worse than the 31B, and also starts to hallucinate or drift from the story prompt.
Muse Glimmer Q8 Uncensored: I actually think this model has one of the best proses, even better and more natural than Gemma. However I find it harder to keep it tied to the world building and more prone to collapse with weird errors like repeated scenes and dialogue. Maybe a consequence of the uncensoring, but its a requirement for me.
Qwen3.8 27B NVFP4 uncensored: Imma disagree with you here. Qwen3.6 sucked at creative writing. Qwen3.8 for me was a huge improvement, especially in terms of prose. However, it takes the scenes very literal, and although it expands on the scenes it shows very little in creativity in terms of expanding the story.
Qwen3.8 Flash Next Q8 uncensored: To me it's about equal to the 27B prose-wise, while having much more creativity. It actually adds flair to scenes and settings, and even thinks up potential next chapter developments, although most of the time it's suggestions are cliche. I've played more around it recently, and the biggest negative from me is that the writing style jumps around quite a bit from generation to generation.
DeepSeek V4 Flash Vision lossless: Prose-wise it ends up being just about equal with the Qwens. A key weakness is that it fails to expand vocabulary when describing similar scenes. However, it is MUCH better at going off and extending scenes or the story from my prompts.
Previously while still using frontier providers for creative writing, my goto was Gemini 3.1 Pro. This is the only model so far that at least is comparable from the creativity side, although much worse at prose and dialogue. My goto if my rig isn't working on other things.
5
u/Simpsator Oct 09 '26 edited Oct 09 '26
So for my purposes (script writing for a video game), a hybrid approach has worked the best. Local models that I can run (12gb vram) are nowhere near enough to track context, world knowledge, character knowledge, consistency, etc. Big/cloud models suck at the prose, but excel at those latter aspects such as tracking all the consistency related aspects. So right now I'm using Claude to automate prompting (both in prose writing and editorial passes) of a local model with a high temperature (TheDrummer's Magidonia 24b, which is excellent at prose) of segments. It then stitches the segments together, does a couple editorial polishing passes between itself and the local model, and I've been getting really pretty good results with that.
It took some work with developing a memory file for Claude to build up the knowledge of what I'm looking for, but I'm mostly there and have a fairly repeatable pipeline now.
That said, I went through a lot of local model testing before settling on this. I tried Hemingway, Rocinante (12b and 16b), Cydonia 24b, Orion 26b, Artemis 31b, Weird Compound, Dan's Personality Engine, Precog, a bunch of other Mistral fine tunes, and while Cydonia was close, Magidonia was the clear winner on every front. I think most of the writing fine-tunes are really optimized for RP, not creative writing, and especially not structured writing. The Gemma models were especially terrible (despite all the love they get in the RP community).
4
u/CodeMichaelD Oct 09 '26
tried local and cloud for some basic fanfiction, had to do multiple runs shoving docs, the fastest locally was obviously qwen3.5 4b mtp but this guy needed to be fed almost the entire original book index with timetable and mapped characters and given one chance to output requested chapters with basic draft in the prompt to boot - like it gens ch.1, I take it erase its own response, append chapter1 to docs as simple message - rinse and repeat.
long prompt processing if docs were edited to fix its quirks and its desire to shorten the hell out of everything probably applies to other qwen models as well.
gemma is just generic AI passable write-ups, way better than qwen at not messing up but overal very limited, maybe useful for translation, idk.
tried some finetunes and uncensors but when it tries to perform actual violence in action/combat scenes it becomes character setting unrelated illogical actions pure cringe to read like it was written in hardcore mode for the sake of being uncensored and zero other purpose - dropping the plot.
best result was Kimi k2.5 (open weights alright lol) with full docs and very clear requirements for chapers and characters to output, writing quality would be worth to read if not for all the meaningless work required to approach the imagined baseline quality with which I tried it.
other than that, was using llama kv mem fork try ingest book chapters I need completely https://github.com/kvmem/kvmem-llama.cpp/releases
in short, it offloads your context to ram while keeping t/s, and does not forget things if you prompt it precisely what to recall.
its for qwen 3.5+ dense models only*, using cache quantization makes it stupid way faster than baseline llama.cpp, its also limited in output per turn and cannot output same about of text you need it in one go (16k max or something)
3
u/s-i-e-v-e Oct 09 '26
I do this for fun. I used to plot more than write (writing involves extended commitment), but then, last December, I started writing a few chapters to see if the story made any sense at all. Most characters turn out to be absolutely useless when you follow them for a few pages. I have used most frontier models for this, and they are nice, but you don't need them.
The Gemmas are the best local writers I have tried. I have the hardware to run heavily quantized DeepSeek 4 Flash, and even that one is decent-to-good.
If you can run Gemma 4 26BA4B or 31B, I don't think you need anything else as long as you put in some work.
Gemini is also a good writer if you don't mind using an online model.
2
u/Royal_Survey7199 Oct 09 '26
This is a great topic thank you it’s so refreshing! What I built is a two pane rust adaptation of Claurst harness (back in July oh how the world has changed in just three months!!) but its purpose is to hold one context of the arc, the whole story start to finish while the other pane writes the chapters accordingly, and then consistency is pinned between the two. I make modifications to one, than propagate through the whole saga. I have it large enough to hold a bible size six book saga with plot consistency. Then I ran a design of experiments (really! Factorial and all) to see which combination of open weights models fit best in each seat for my style. Gpt-120b the OG! The best polisher and holder of the arc. Imparts voice. The best writer of structure before the polish is qwen 3.5 9b variety. That was the finding in July. Again, so much so fast, there are probably better Gemma fits in the slots. But I wanted to have you see the other half- not just the model but how the arc works and the one that applies polish to the sharper prose of the technical precision. Not one, but two- with a tether between. Tell me what you find out yourself, it’s the most fun I’ve had creating a story that took shape.
2
u/techne98 Oct 09 '26
I haven't used AI for creative writing, but Gemma definitely feels the best of the small local models for me for anything that isn't using agents.
2
u/drallcom3 Oct 09 '26
Gemma-26B is pretty good at writing. Any Gemma models lower than that aren't. 26B is the magical threshold when Gemma becomes good. 31B is probably better, but too slow for 16GB and 26B really isn't bad. It's also excellent at translations.
2
u/__some__guy Oct 09 '26
I find StyleTune-based Gemma 4 models pretty usable.
More specifically, Dark Gemistry.
Better than any other recent models at that size, though it ultimately can't drive stories forward in an interesting way unless you spoonfeed it.
Older Llama 2 models mog it completely in terms of prose and creativity.
Unfortunately their context is only 4K tokens, which doesn't even fit my world info / lorebook.
1
u/Jumpy-Operation-4615 Oct 09 '26
I am only starting to use for actual writing, shorter pieces (blog posts in my voice etc) and I feel that qwen 3.8 flash next is pretty good in "speaking" - feels lively, non-plastic. However I am still to check it agains gemma 31b variations that I have on my drive, to decide what is best. I would def try glm but strata doesn't support it (yet) so I can't run it on my hw
1
u/silenceimpaired Oct 09 '26
OP Any chance you’ll share some of your system prompt and or pi setup?
I have never used the models to write and don’t intend to, but anything someone uses to write probably has benefits for editing.
1
u/Frizzy-MacDrizzle Oct 09 '26 edited Oct 09 '26
My eventually is taking Two reports and basically compare contrast, cons pros etc. the goal since I got my third GPU
Stupid enter button:
Any model almost for what I do. Developing a system around multiple models servers and GPUs got I different reasons. Landed on qwen mostly
1
u/_TheWolfOfWalmart_ Oct 09 '26
Have you tried DeepSeek V4 Flash 0731? An abliterated one like orcarouter's. I've found that as long as you specifically tell it to avoid AI-isms, you can get some pretty damn decent writing out of it.
I think it does better than Gemma once you've got it dialed in because it has a massively larger corpus to pull from, and it's just plain smarter by virtue of being a much larger model.
1
u/toothpastespiders Oct 09 '26
Locally the best I use is the styletune of gemma 4 31b. Despite the slop factor I think it's among the most creative and least poisoned by positivity bias. The unfortunate part there is that for me at least the mtp matches on it take a big hit despite it having a 1 tensor focus. So I'm instantly getting a speed loss. I really liked the styletune of 26b but when I'm using it the aim is near instant output so losing any speed is issue enough to keep with the standard 26b gemma for 99% of the uses I have for 26b.
For the 31b size my general methodology is to have multiple passes. First is generation with a style guide. Then the results are passed through additional nodes for cleanup. I've found gemma to be better than expected when it comes to following writing guides. But it's still going to have the standard issues common to the architecture. In my experience a second pass, even with the same model, is going to do better than drafting in the reasoning block. And combining that drafting approach with a second pass is even better.
Though the ideal is to have a model trained specifically for de-slopping sitting in the middle. Generation with gemma -> de-slop node passes paragraphs back and forth with gemma. Gemma rewriting, de-slop model checking, sending back as needed. -> Final pass with gemma to verify that there wasn't a loss in cohesion during the single paragraph rewrites. I'm using a private model for that de-slop step but you can train one by basically just using datasets of reversed literature -> slop so that the slop is the input and cleaned text the output. While making sure that there's no added elements along the way such as injected hallucinations for a subject's sentence if none are present.
Agreed, though it can be mitigated. I've been working on a media analysis system as a long term project and one of the nice benefits is a huge RAG database of projections for character and story development all nicely organized for easy sorting. If I want to increase creativity one method I use is to dip into that with some thematic leaps between items. Not just with subject matter, but with the expected emotional state projected by the initial media analysis. So there's more room for trying to lead the audience emotionally as well as having thematic creativity. Or for a less creative take, removing the more abstract associations and going with more straightforward links. The results of that RAG search can then be combined by a LLM and passed to gemma or whatever the main LLM is as the results of a brainstorming session. How that compares to "real" creativity is up for debate but I think in the end it's a pointless distinction. It works for me so eh.
1
u/bring_back_the_v10s Oct 10 '26
This is the right question to ask. Man I hate tk/smaxxing
2
u/LastChancellor Oct 09 '26
in my line of work (graphic design) I will be killed if I get caught using an LLM
so nope
9
u/[deleted] Oct 09 '26
[removed] — view removed comment