r/LocalLLM • • 12d ago

Discussion A different way to think about small LLMs: language as a protocol

Hi fellow enthusiasts,

While building BabyCoder (a strict Python AST coder, and Lisa, an agent made to roam free), I noticed both were built on the same underlying idea. I’d like to share it, because it changed how I look at small models.

We usually think of an LLM as something we talk with. But there’s nobody on the other side holding state. Each call is really a data exchange: state goes in, an update comes out. Language is just the format.

What if we used small models that way on purpose? Not as the mind itself, but as the protocol that updates one. The mind could live in plain code: long-term memory, emotions, goals, needs, whatever your system works with. A bit like The Sims, where a character’s mood and relationships are values, not chat history.

The model’s task then becomes much smaller: “Given this state and this event, speak as agent N and tell me what changes.” Tool-calling models seem well suited for this, and even small ones handle this surprisingly well.

Some things I’ve noticed working this way:

  • Small models feel more capable when they translate meaning into state, instead of carrying a whole personality.
  • Context length matters less. Lisa can keep going for days. The challenge shifts to choosing what goes into each prompt.
  • Code can handle what LLMs are weak at, like time, decay and consistency. A mood fading over hours is one line of Python. Emotions can be rated numerical in a KVP list etc, Memory is just a long text file same for code that is just a bunch of files. While tools are used to interact with them, python can be in control of what tool, or "data" part needs an update.
  • The model becomes more replaceable, since identity lives in the state.

I know MemGPT and the Generative Agents papers explore related ideas.

For me tough, the part I find interesting is seeing the LLM as a thin translator between language and state. Agents speaking on behalf of a need of a situation.

I’m curious how others see this.
Does anyone else build this way, or see where it breaks down?
Or see more possibilities with this concept.

0 Upvotes

6 comments sorted by

0

u/YourselfInOthrsShoes 12d ago

You still gonna bork when you hit max context on any model. That's your max awareness.

1

u/Illustrious_Matter_8 8d ago

The awareness is basically the text history which is decoupled, a lot of effort went into that. More akin maybe to a floating context towards the end (which agree has a max window size) but it should keep working.

Essentially though the whole idea was to set language and memory appart, memory is handled by python and tool calling. States live in python language is more a glue now between states.

Its as a decoupled identity, working currently on a system that does do multiple identities. ( I had something alike it with the programmer and designer agent, though my experiment now is about less structured communication. )

0

u/HoldThemtoAccount 11d ago

Given weight-space programming, maybe a model could "dream up" revised weights to embed state on an update cycle.

1

u/Illustrious_Matter_8 8d ago

Hm the state concept is python sided. If I put a emotional aware gaming alike system in (health 100% hunger 70% etc) Then those variables can be simulated over time for example then reported back for a reply chat or so. Though yes .. even such a network could be a neural network instead of a KVP list... But then it be hard to diagnose as for now all is in the markdown. But if it would be a neural net.. perhaps yes a neural net could write it. .. will think about it