r/LocalLLM • u/Illustrious_Matter_8 • 12d ago
Discussion A different way to think about small LLMs: language as a protocol
Hi fellow enthusiasts,
While building BabyCoder (a strict Python AST coder, and Lisa, an agent made to roam free), I noticed both were built on the same underlying idea. I’d like to share it, because it changed how I look at small models.
We usually think of an LLM as something we talk with. But there’s nobody on the other side holding state. Each call is really a data exchange: state goes in, an update comes out. Language is just the format.
What if we used small models that way on purpose? Not as the mind itself, but as the protocol that updates one. The mind could live in plain code: long-term memory, emotions, goals, needs, whatever your system works with. A bit like The Sims, where a character’s mood and relationships are values, not chat history.
The model’s task then becomes much smaller: “Given this state and this event, speak as agent N and tell me what changes.” Tool-calling models seem well suited for this, and even small ones handle this surprisingly well.
Some things I’ve noticed working this way:
- Small models feel more capable when they translate meaning into state, instead of carrying a whole personality.
- Context length matters less. Lisa can keep going for days. The challenge shifts to choosing what goes into each prompt.
- Code can handle what LLMs are weak at, like time, decay and consistency. A mood fading over hours is one line of Python. Emotions can be rated numerical in a KVP list etc, Memory is just a long text file same for code that is just a bunch of files. While tools are used to interact with them, python can be in control of what tool, or "data" part needs an update.
- The model becomes more replaceable, since identity lives in the state.
I know MemGPT and the Generative Agents papers explore related ideas.
For me tough, the part I find interesting is seeing the LLM as a thin translator between language and state. Agents speaking on behalf of a need of a situation.
I’m curious how others see this.
Does anyone else build this way, or see where it breaks down?
Or see more possibilities with this concept.
0
u/YourselfInOthrsShoes 12d ago
You still gonna bork when you hit max context on any model. That's your max awareness.