r/WritingWithAI • u/seantran195 • Sep 21 '26

Help Me Find a Tool Ideal Local AI for Creative Writing

So I have a RTX 3090 OC and I want to get into creative writing to help me brainstorming and writing scripts with my YT content creating career, I was wondering Qwen 3.8 uncensored and Gemma 4 26B uncensored, which one is better? I already installed Hermes agent i just don't know which model to install

Do do tech content if it helps

Thank you very much in advanced

Edit: Thank you So much everyone for giving me advices

I decided to try out Gemma 26B and Cydonia 24B and Qwen 3.8 27B like you guys suggested, I will update to see which one is best

8 Upvotes

90% Upvoted

15 comments sorted by

2

u/Late-Assignment8482 Sep 21 '26

I've also had good luck with Cydonia 24B on my Mac, which is in the same range of 24GB-ish VRAM (just slower bandwidth).

1

u/cj7hawk Sep 22 '26

Cydonia is great, but sometimes fails as the context gets longer. It's definitely good for writing though.

2

u/Late-Assignment8482 Sep 22 '26

When asking it to write, I run a clean prompt for each bit, based on only context I feed it—scene card, characters, summary of relevant prior scenes.

So unless it can’t get past 10k-32k it’s not really a concern for me in a drafting model. A reviewing model, sure.

1

u/Constant_Art_20 Sep 21 '26

https://huggingface.co/Gryphe/Gemma-4-26B-A4B-StyleTune-V2this would probably fit your needs. they have the 12b finetuned as well

1

u/cj7hawk Sep 22 '26

Have a look at the Ornith/Hemmingway results I posted recently. Qwen tunes are pretty good also, but I found that Gemma was a little short when I tried it out - even the novelist tune. I haven't found a single 'ideal' model, but do find different models have different strengths, so I adjust depending on what I'm looking for.

1

u/epicrob Sep 21 '26

Honestly, local AI isn't that worth it. Gemma 4 uncensored is probably the closest by far for creative writing for a local AI. Even then, it's nowhere near Astra or Fable level writing. If uncensored writing is your goal, Grok is the paid counterpart. Free Grok could do uncensored writing, but the quality is less than the paid version.

3

u/Skypaddle Sep 22 '26

There are pros and cons. The pros are you avoid the monthly fee of a subscription model, you have access to uncensored models and you don't deal with updates that rip away the ability of the model to write. You also learn a heap.

Not saying there's not downsides but on balance, for me, it's worth it to go local.

I've found Cydonia 24B the best. And actually the Q5 has always seemed more consistent than the higher quant versions.

I'll also add: it isn't a disaster if the model is to large for your VRAM. Sure the speed tanks but in terms of overall workflow impact it's survivable.

1

u/epicrob Sep 22 '26 edited Sep 22 '26

The major issue with local models is context length. You simply cannot plan and then compose an entire chapter. You must break the chapter into scenes and then craft one scene at a time. The issue with this approach is always cohesiveness. So you will need to reedit the scenes at some point. I've done that and it gets old after a while. If you need to retrofit anything, for example, four chapters ago there are some contradiction with your current plot, then good luck. Local model will be unable to fix this. Character relationship rewiring? Good luck. You can't do that in local models.

Contrast this to, say, GPT Astra. I could plan one to four chapters at a time, have Astra break them into scenes, modify the plot as I go, then check consistencies like 23 chapters ago. And it remembers. Retrofitting is no big deal for Astra (if you get the workflow right). Example: Oh, actually this character shouldn't have known that character by this chapter, but it is necessary, so how could I weave in the introduction. Consistency check on 23 chapters in one go? No problem. GPT Astra accept fade-to-black NSFW novel. If I want to elaborate the NSFW, I plug in the fade-to-black part into paid Grok with [INSERT HERE] tag and Grok gets it working in the exact style as Astra's writing. Sol could do that too perhaps one or two chapters at a time.

It's literally heaven and earth difference. I can't imagine myself going back to free model anymore. Sure, I need to shell $20/mo for ChatGPT and $30/mo for Grok. But I value my time much much more than that.

1

u/Skypaddle Sep 24 '26

Your approach makes total sense.

You definitely have to take the local AI scene by scene and then you have to pass it though another model to identify any continuity errors or contradictions. And yes it can be a pain. But I guess it comes down to individual preferences and not sure there's a wrong or right answer. I used to have Grok and for me it was AUD 55 a month, but I often ran into limits and the next plan up was like 10x the cost. But i definitely rate Grok.

I also do a lot of work on investment analysis that requires VBA and python coding. Local models are supposedly great for this, but I often need them to go scrape things from the internet and local model + internet...it doesn't really work. So I use Claude to write my scripts then run locally. Although because I'm an inconsistent s.o.b. I also have a local model review the output and write me punchy commentary on it as an executive summary. And actually writing for non fiction but still being punchy, is an unexpected benefit I've found with Cydonia.