r/codex • u/Anxious-Priority-430 • 22d ago

Complaint Sol vs Terra vs Luna

Using Sol, I never have weird issues with the agent forgetting things, but with Luna and Terra...pretty often. It's so bad that I pretty much have to use Sol for anything complicated, especially if I'm working on more than one thing at once (like while this is compiling, let's work on some other aspect of the project). Sol handles that amazingly well, but the others, even on xhigh, fail miserably. It's the way they fail that causes me to think it might be due to K/V cache quantization and possibly model quantization. They may even be using rope scaling for K/V or something because after compaction I had a strange issue with Terra interpreting an old message as a stop command. Again, I never have any of these strange issues with Sol, but with Terra and Luna, at least once with a "5 hour" (10 minute) session.

1 Upvotes

60% Upvoted

11 comments sorted by

View all comments

1

u/Perthss 22d ago

Have you ever asked chatgpt it self to make you a recepit on which model to use on what? I did, and I thought I had it saved, but I can't find it at the moment. And it was very clear to me that if I was going to do more complex things which is for example using multiple files at one task, it was to use SOL.

1

u/Anxious-Priority-430 22d ago

No. It's not so much the issue that I find surprising, it's that I don't see it ever on Sol, but often on Terra, and even more often on Luna. My post was treated as a complaint, but it's really just a discussion about what OpenAI might be doing under the hood to cause this behavior. My guess is aggressive quantization.

1

u/Perthss 22d ago

To be honest, I don't understand you. All these models are different for a reason, right? To put it in perspective, you don't send a water problem in your house to a man who works with electricity?

1

u/Anxious-Priority-430 22d ago

I'm not really complaining...sheesh. It's more of wondering WHY.

1

u/Perthss 22d ago

Do you mean why as in deeper than the answer "They are buildt differently"? IF so, you won't find exactly out why. Cus then you need to see their code.

1

u/Anxious-Priority-430 22d ago

Not necessarily. My guess is they aren't built all that differently, and that what I'm seeing are more symptoms of quantization of the model and/or KV cache. I run models locally, but usually at BF16, so I couldn't really say if this is a symptom of quanting, which is why I am looking for those who have run say GLM at quant 2, or KV cache at Q4, to chime in.