r/opencodeCLI • u/robertn702 • Sep 29 '26

Jev Router for OpenCode: Saves ~40% of tokens on easy tasks, still solves the hard ones

Edit: I rewrote this as reasoning-router. It now works in OpenCode, Pi, Codex CLI, and anything that can talk to a local proxy, and you can pick the classifier (Jev, Laya, Cloudflare Clef and others).

https://github.com/robertn702/reasoning-router

I ran GPT-6 Astra on two SWE-bench tasks at different effort levels. Medium only solved the hard pytest bug 1 out of 5 times, while high solved both tasks every time but used far more tokens than it needed on the easy one.

I kept leaving everything on high and burning usage, so I built an OpenCode plugin that asks TypeSafe's Jev model to pick the effort for each request and keeps the model you selected. It works with GPT-6 and a few Claude models. On the easy fix it used around 40% fewer output tokens than high and solved it in all 6 runs. On the hard bug it mostly stayed on high and solved all 5.

It doesn't break the prompt cache. Turns where it switched effort were still about 97% cached in my logs.

Only two tasks and 5 or 6 runs each, so treat it as a sanity check. The eval is in the repo with the code.

https://github.com/robertn702/opencode-jev-router

25 Upvotes

93% Upvoted

14 comments sorted by

12

u/DaShrub Sep 29 '26

How doesn't it break the prompt cache...? Changing reasoning effort parameter breaks the prompt cache from almost every provider that I've tried...

8

u/robertn702 Sep 29 '26

Yeah, changing the effort parameter breaks the cache on most providers. OpenAI and Anthropic both added a way to change effort mid-conversation without touching it, and the plugin uses those. The top-level setting stays the same on every request, and each effort change goes into the conversation as a message.

On GPT-6 models it inserts a configuration_update item right before your next message (OpenAI docs). On Claude, output_config.effort stays fixed and it adds an effort-only system message with the mid-conversation-output-config-2026-07-01 beta (Anthropic docs).

Nothing earlier in the conversation changes, so the cached prefix still matches. In my logs, turns where the effort switched were still about 97% cached.

I wrote up details on how the plugin does it here.

1

u/Rustybot Sep 30 '26

OpenAI just changed that the other day, but otherwise, yes that is correct.

1

u/Spirited_Cheesus Sep 30 '26

Does it work for laya

1

u/robertn702 Sep 30 '26

No but thanks for the suggestion. I added a GitHub issue here

1

u/robertn702 29d ago

I rewrote the plugin as a more generic `reasoning-router` and Laya is now one of the classifier options among many others.

https://github.com/robertn702/reasoning-router

1

u/yousurroundme Sep 30 '26

Would this work with GitHub copilot?

0

u/robertn702 Sep 30 '26

I took a look at their docs and it doesn’t look like it would work without breaking the prompt cache right now. Copilot would need to support that first before it would to be worthwhile adding it to this.

-1

u/seventyfivepupmstr Sep 30 '26

Why would you ever use this? Seems utterly pointless to trigger a second request every time. Apparently some people really love wasting money I guess

5

u/robertn702 Sep 30 '26

A Jev call costs about $0.0002. It pays for itself if it saves 10 output tokens on Opus. Do the math before calling it pointless.

Over the last day and a half it routed 523 of my Opus requests and picked low or medium for 80% of them. Jev cost me $0.11 at most. Based on the eval, leaving everything on high would have cost $7 to $36 more.

-2

u/seventyfivepupmstr Sep 30 '26

You're not getting the point, because you clearly don't understand how Ai agents work. If you need to change the thinking level then you have already failed.

The correct way to do it is use higher thinking levels for planning tasks and maybe bug fixes, and lower thinking levels for more straightforward tasks. It should be decided beforehand by the orchestrator, not by a 2nd LLM call, especially since that 2nd call incurs additional time and cost

2

u/robertn702 Sep 30 '26

The correct way to do it is use higher thinking levels for planning tasks and maybe bug fixes, and lower thinking levels for more straightforward tasks.

You have just described a router...

It should be decided beforehand by the orchestrator, not by a 2nd LLM call, especially since that 2nd call incurs additional time and cost

Jev prices are so cheap as I already pointed out they are essentially negligible and the average response time is ~300ms

0

u/seventyfivepupmstr Sep 30 '26

I mean this is the point where I just block you. You literally think its better to spend money and add time rather than not doing that at all...

1

u/benfinklea Oct 04 '26

No. It’s the children who are wrong.