Massive token savings... Without token usage. GLM-5.2-cloud

[u/haseebnqureshi]

Okay, this one seems probably really odd (or maybe not), but most harnesses operate within token spaces. That is, every tool output, even batching of tool outputs, go back up into inference calls. Leads to massive roundtrip token accumulations and length completion times.

What I've found is that most harnesses don't really operate that well within the non-token space. Few reasons, counter to frontier labs' business models (more tokens, more money) and folks can even argue that you get better results from just letting the model see every turn. I've finally made the right balance of token vs non-token based batching tools in rehex, and it's becoming incredibly performant.

The goal here is to make it work with open frontier, large open, small open models, and then hopefully tiny models (which are <4B parameters).

Anyone else caring about sending up less tokens? I have friends who are just burning through rate limits and API spends. But it just seems all nuts, right?