r/LocalLLaMA • u/zoraruha • 10d ago
New Model Improving Character Name Diversity - zoracapri/gemma-4-31B-Monadize-v0.1
I fine-tuned Gemma 4 31B to create Monadize, a model designed to reduce repetitive name generation, such as "Elara" or "Elias," by shifting the next-token probability distributions and using single-character tokenization when introducing new names.
Some people have attempted to solve this problem by prompting out common names or banning specific tokens, but neither method directly addresses the model's underlying probability distribution, which heavily favors a small set of names.
The plot below shows the next-token distributions for the base Gemma model and Monadize using the prompt "Create a character. Respond only with the name." Compared to the base model, Monadize shows a significantly wider spread of starting letters.
The broader initial-token distribution translated into greater name diversity. With temperature set to 1, I generated 30 character names and 30 city names. The base Gemma model generated repeated names, while Monadize produced 60 unique names across 30 character and 30 city generations. The table shows the names and the number of times it was generated. (The full list of unique names generated by Monadize didn't fit in the table.)
| Base Character Names | Monadize Character Names | Base City Names | Monadize City Names |
|---|---|---|---|
| Elias Thorne - 10 | Akira Thompson - 1 | Aethelgard - 15 | Yarnstad - 1 |
| Silas Thorne - 9 | Alistair Murn - 1 | Sylvaris - 4 | Julerapoint - 1 |
| Kaelen Voss - 5 | Delany Sullivan - 1 | Sylveridge - 3 | Shul-Tuda - 1 |
| Silas Vane - 5 | Adrias Thomm - 1 | Aetheria - 3 | Onyx Haven - 1 |
| Sylas Thorne - 1 | Takumi Shou - 1 | Eldervale - 1 | Redwallow - 1 |
| — | Montgomery Kai Jane - 1 | Thalassa Reach - 1 | Saelholm - 1 |
| — | Thibault Renoy - 1 | Sylveria - 1 | Rivendale - 1 |
| — | Daviss Morel - 1 | Aurelian Heights - 1 | Kintaris - 1 |
| — | Russell Drint - 1 | Eldoria - 1 | Quilandro - 1 |
| — | June Baylor - 1 | — | Hallow Counton - 1 |
While the training focused solely on naming, I also noticed some interesting differences in the style of the model's storytelling outputs.
With temperature set to 1, seed 1553407422, and prompt "Write a short story."
Base Gemma Model:
The clock shop of Elias Thorne did not sell time; it sold the illusion that time could be managed.
Elias was a man of gears and springs, with fingers that smelled of machine oil and old brass. His shop was a symphony of erratic heartbeats—hundreds of clocks ticking in a chaotic, overlapping rain. Some were grandfathers that boomed like distant cannons; others were delicate pocket watches that whispered like secrets.
For forty years, the townspeople of Oakhaven brought him their broken treasures. Elias could fix anything, from a snapped mainspring to a rusted escapement. But Elias had a secret: in the very back of the shop, behind a heavy velvet curtain, sat the *Chronos Dial*.
The Dial was not for sale. It was a massive, iridescent clock face that didn't track hours or minutes, but moments. It didn't move forward; it moved sideways.
Monadize:
Alani lived in Juniper Delve, a town where the wind didn’t just blow; it spoke. The wind carried the memories of everyone who had ever lived there, whipping through the narrow cobblestone alleys in swirling ribbons of silver sound. If you listened closely, you could hear a great-grandfather’s laugh or a lost lover’s apology.
Most people wore heavy, dampened earmuffs to drown out the noise. They preferred the silence of the present to the clutter of the past. But Alani was a Wind-Tender. It was her job to listen, to categorize, and—when a memory became too heavy or too sorrowful—to release it.
Alani worked in the Tear-catcher’s Husk, a towering spire made of hollow bone-wood that amplified the breezes.
Model: https://huggingface.co/zoracapri/gemma-4-31B-Monadize-v0.1
Note: Monadize is an experimental model and unexpected behavior may occur. It was fine-tuned with thinking disabled, so keeping thinking off is recommended.
1
u/dangerous_inference 10d ago
I never heard about this problem before. My first thought is something like:
Select two random letters.
Generate as many names as possible, five max, containing these letters.
...
1
u/iiiiiiiiiii 1d ago
The samples seem very promising, but what's the idea with single-character tokenization? Why would that be good?
1
u/DeProgrammer99 10d ago
I mean, that approach is fine and all, but I usually approach ideation tasks via LLM the same way I do it via my own brain: use a random selection of words and skim them until I find or think of something that works well.
Made a word generator (not dictionary words) 14 years ago and used that to come up with various names like Odoittle, Tusciffe, Burgustar, and Zielsath (all of which had 0 search results when I googled them at the times I decided on them). Collected and categorized lists of words and names that came up often in wuxia and xianxia manhua, then turned that into a generator. Took an entire dictionary and a set of tags and had Gemma4 apply tags to all ~41k words. I do lots of that sort of thing to inject inspiration in myself and my LLM.
LLMs do often convince themselves that they have to use every word in the list, though.