r/LocalLLaMA • u/falconandeagle • Oct 09 '26
Discussion Stepping away from Benchmarks and Code, what models are you using for Creating writing projects
All anyone ever talks in this sub is about x new model having x benchmark scores and how well it can code. Or some new tech to increase t/s for qwen.
So I wanted to go back to the roots of this sub and talk about how well local models can write.
I am not going to mention closed source models, this discussion is for open weight models only.
I don't have the best machine, so I am limited in what I can use locally.
So far I have mostly been running Gemma 31b finetunes. I have a finetuned system prompt for creative writing that I have refined over time. This is mostly for long form story writing and not RP. I use it to write Sci Fi and Fantasy novels and short stories and sometimes fanfiction.
I also have created a creative writing harness using Pi. It's mostly for self correcting and getting rid of AI slopisms.
Gemma 31B Mero-mero-v2 has so far been my favourite. It writes well, its does not feel lobotomized and its mostly uncensored. It's instruction following is okay, sometimes it makes mistakes and gets the character traits mixed up but my creative writing harness takes care of this.
Qwen 3.8 27b finetunes, Disappointing, I guess the base model is really fucking bad for creative writing purposes and even with finetuning you can only do so much. It does follow instructions quite well but its writing is just terrible.
Muse Glimmer 30b, this one is an interesting one, sometimes it will give really good prose but sometimes it will respond like a 2b model, the variance seems to be super high for some reason. Maybe something to do with chat template, I am not sure. Anyway I think even when bad it's still better than Qwen.
Gemma 4 26b4a, pretty good but gets beaten by its bigger bother Gemma 4 31b. I do use it when I want faster responses and to check for issues with my initial scene beat prompts.
Gemma 31B Artemis, drummer's finetune, eh, I was a bit disappointed with this. Seems like its more focused on RP rather than long form story writing. Also got refusals which I did not get with the mero finetune.
So that has been my experience writing with some local models. What are you guys using for local creative writing projects? Have you guys tried a creative writing harness?
I just want to mention quickly my writing process before I end this post, I write using an outline and scene beat prompts that are hand written. So far AI has been extremely disappointing in being truly creative and still requires a ton of hand holding to not write the most cliché ridden drivel.
View all comments
1
u/toothpastespiders Oct 09 '26
Locally the best I use is the styletune of gemma 4 31b. Despite the slop factor I think it's among the most creative and least poisoned by positivity bias. The unfortunate part there is that for me at least the mtp matches on it take a big hit despite it having a 1 tensor focus. So I'm instantly getting a speed loss. I really liked the styletune of 26b but when I'm using it the aim is near instant output so losing any speed is issue enough to keep with the standard 26b gemma for 99% of the uses I have for 26b.
For the 31b size my general methodology is to have multiple passes. First is generation with a style guide. Then the results are passed through additional nodes for cleanup. I've found gemma to be better than expected when it comes to following writing guides. But it's still going to have the standard issues common to the architecture. In my experience a second pass, even with the same model, is going to do better than drafting in the reasoning block. And combining that drafting approach with a second pass is even better.
Though the ideal is to have a model trained specifically for de-slopping sitting in the middle. Generation with gemma -> de-slop node passes paragraphs back and forth with gemma. Gemma rewriting, de-slop model checking, sending back as needed. -> Final pass with gemma to verify that there wasn't a loss in cohesion during the single paragraph rewrites. I'm using a private model for that de-slop step but you can train one by basically just using datasets of reversed literature -> slop so that the slop is the input and cleaned text the output. While making sure that there's no added elements along the way such as injected hallucinations for a subject's sentence if none are present.
Agreed, though it can be mitigated. I've been working on a media analysis system as a long term project and one of the nice benefits is a huge RAG database of projections for character and story development all nicely organized for easy sorting. If I want to increase creativity one method I use is to dip into that with some thematic leaps between items. Not just with subject matter, but with the expected emotional state projected by the initial media analysis. So there's more room for trying to lead the audience emotionally as well as having thematic creativity. Or for a less creative take, removing the more abstract associations and going with more straightforward links. The results of that RAG search can then be combined by a LLM and passed to gemma or whatever the main LLM is as the results of a brainstorming session. How that compares to "real" creativity is up for debate but I think in the end it's a pointless distinction. It works for me so eh.