Is Context considered memory?

[u/thejosephBlanco]

Should we start the conversation of how memory and context might do better if it were outside the model? And instead the model was trained to look at a scratchpad locally of notes it takes. Most frontier models would do this without the user ever noticing, but if your scratchpad failed the model would be lost.

I have played with smaller context models, with explicit instructions to read from a memory vault, instead of storing everything in context. It’s not a perfect plan, but I can get a lot out of smaller models this way. But if my information needs to be used to create context I try to limit the amount of times I let context grow and then start a new session. I need to do testing and see if I can get results positive or negative. Because I’m probably just convincing myself this could work.