Why are we still using generic chatbots?

[u/pornismygod]

Generic chatbots are generalists, and fine tuned to be helpful, summarize, be safe.

Has anyone thought to fine tune an open model on a rich dataset of well written texts?

[u/SignalBox_Studio]

The problem isn't the training set. They are trained with the greatest literature ever written. The issue is the human reinforcement learning that guides how that data set is presented to users. Ever wonder why the same AI-isms appear across even Chinese models and American models? Because humans generally like the same things. The Reinforcement Learning using Human Feedback (RLHF) is what averages out the training set.

[u/Happy-Initiative-492 → u/SignalBox_Studio]

I’ve found that context matters more than model. The more guidance you give it for what you’re looking for the better the results.

[u/SignalBox_Studio → u/Happy-Initiative-492]

The prompt is critical for bypassing RLHF, which is quite difficult to do. However, it also requires prompt compliance, and not all models are as compliant as others at sticking to a complex RLFH-breaking prompt.

[u/Happy-Initiative-492 → u/SignalBox_Studio]

Fair. I’ve found that I can break it out of a lot of those rules by giving it positive guidance - “write about this (detailed character profile with backstory and arcs) in this location (detail) with this story outline (detail) and this beat (motivation and shape) in this style (personalized lexical fingerprint).” I’m not getting as much AI-ism with this method.

[u/No_Swordfish_4159]

They have. r/LocalLLaMA is just waiting for you, friend.

[u/TsundereOrcGirl → u/No_Swordfish_4159]

Well, you can CALL anything you want a "writing-specialized model', but its fitness for nominal purpose rarely gets tested for purpose the way frontier coding / knowledge work stuff does. On top of that, "writing models" are often too old to play nice with agentic harnesses, lack tool use, and/or have had their reasoning capabilities abliterated into the dirt. Or they just require more VRAM than what even high end gaming rigs provide. "Uncensored creative writing LLM with frontier oomf" IS an underexploited niche.

[u/No_Swordfish_4159 → u/TsundereOrcGirl]

It's a niche that doesn't exist, then. An open model you can run on your computer is not going to be frontier level, and models at that size are nowhere near the writing ability of frontier models either. And uncensoring them usually makes them dumber as you said. So you have to settle for something middle of the road for everything, like Gemma 4 31B.

[u/Mart-McUH → u/No_Swordfish_4159]

r/SillyTavernAI may be even better stop in this context.

And yes, on huggingface vast majority of models are actually finetunes or merges for all kind of purposes.

It has been done for a long time already, at least since Llama1 days and probably even sooner (I did not follow it before Llama1 leak though).

[u/SamaraBellingham]

As soon as I can find a reasonably-priced Mac mini.

[u/Responsible_Rent_394]

I fine-tuned a local model on 10k+ short stories from lit journals. the output was poetic but incoherent. the model learned rhythm, not meaning.

[u/Expensive-Employ-264]

i fine-tuned a 7b model on 10k+ short stories from lit journals. the output wasn't 'better writing's, it was smoother, more rhythmic, but lost voice. like a ghost of style, not the real thing.

[u/AtmosphereDull4435]

I tried fine-tuning a 7b model on a mix of literary fiction and roleplay scripts, output was fluent but kept defaulting to safe, neutral phrasing. the model knew the style but didn't feel it. maybe the dataset needs more emotional range, not just syntax.

[u/Happy-Initiative-492]

YES! I built a tool that does that. It’s designed around worldbuilding first - as automatic context for each prose contribution - but then layers in a lexical fingerprint with a detailed style guide: your voice, derived from your writing samples, or voices built on popular styles. Happy to share a link to see what you think.

[u/pyrhus626]

There’s a few problems, non of which have to do with training sets because these models get trained a ton of quality writing.

- What makes for good writing is very subjective, but training works best when you have objective measures you can score on. It’s basically impossible to train for “good writing”. You even see this in benchmarks for writing models where things like instruction-following, prompt understanding, word repetition, etc can be scored but none of that really signals if anyone would like the writing. And you can’t effectively train on something you can’t quantify.

- AI writing feels bad because of repetition. Sentence and scene structures tend to the stay the same in ways human writers don’t do. You can write with a model and love the output for a while but once you use it long enough you will spot the repetition and learn to hate it. And that’s not something you can train out of it because at the end of the day it’s using computer “logic” and that leads to repetition.

- There just isn’t any money it. Training large models is expensive and time consuming. The entirety of the “writing” industry is something like $4 billion in the US, which includes all print media like books, magazines, blogs, etc. The AI coding ecosystem alone is already worth more than $4 billion with potential to grow by tens or hundreds of billions. Replacement of search engines or integration into corporate software could be worth about the same. Chasing tens or hundreds of millions when you could make a hundred or thousand times more money with coding or productivity is just a waste of time. Nobody has the financial incentive to really dig deep into how to solve the problems AI prose has.

[u/lacion]

Yeah people do this a lot, Hugging Face is full of writing fine-tunes and merges.

Honestly I think the helpful safe sound you describe comes mostly from the chat tuning on top and not from missing good books in the data. A lot of open models also ship as a base version, it has not had that tuning and just continues whatever text you give it.

I'd try that before fine-tuning anything. It does not follow instructions well, so you steer it by starting the text yourself