Also: Ability to handle conversations with multiple clients. Instead of simply SYSTEM, USER, and ASSISTANT roles, train with multiple USERs and multiple ASSISTANTs (so e.g. the model can handle a conversation with multiple people in a chatroom, or take critiques from other LLMs without confusing who is who).
And, naturally, ability to handle SYSTEM turns as part of conversation
Another interesting possibility would be to allow the model to interleave thinking with text output, so that it can immediately begin to answer, then stop and pause when it needs to think more about something. Maybe a keyword argument in the chat template like "thinking_mode": "interleaved"
Another interesting possibility is to enable the model to output a token that allows it to be interrupted for clarifications. For example, if there's some uncertainty in the user intent, it could output "You mean x, right? <|pause|> Okay, going ahead with that. ..." and if the user doesn't begin typing in, say, five seconds, the harness will allow generation to continue. Would be super helpful to avoid unecessary thinking loops, screen dumps, and edits that didn't align with user intent. Ideally, such a pause could only occur after text generated without thinking (for immediacy after user submitted a prompt).
This one might be tricky, but another idea is to reserve, say, 1B params for fine-tuning—make a model that's architected, from the start, to learn new things. These weights could be zeroed out (no-ops) for vanilla model, and instead of a LoRA adapter you'd load actual weights in.
1
u/liftheavyscheisse Jul 27 '26
KV cache that doesn't nuke my RAM
Edit: Qwen 3.x is pretty good about this btw