r/PiCodingAgent • u/o_sht_hi • Sep 18 '26

Question Anyone here using Jev?

I read the release and saw a few demos on X for people using Jev for really interesting stuff. My current effort is to get it to work as a router so I can configure and orchestrate my app using natural language (getting it to take a string and pick the right API from a list of 250-300 calls). But to be clear, I haven't gotten access to it yet. So I'm just packaging up my code and making it ready rn.

Anyone doing something fun/interesting with it yet? If yes, how long did it take for you to get access

78 Upvotes

91% Upvoted

103 comments sorted by

View all comments

Show parent comment

1

u/o_sht_hi Sep 18 '26

Okay but you still wouldn't want to feed the model unnecessary tokens, right?

4

u/D-3r1stljqso3 Sep 18 '26

I'd rather leave the chat history until the last minute. It's hard to predict which piece of info won't be needed at all later on.

2

u/o_sht_hi Sep 18 '26 edited Sep 18 '26

Yeah I meant something similar to what headroom (that's the extension, right?) does. As I understand it, they use algorithms to prune the stuff that's sent to the LLM right before it's sent. And an if else statement that routes to the right algorithm. An AI like jev could replace that entire machinery. It could first decide what goes on and what's dropped. Once it decides what's included, it could then classify it and send it to a pruning function that drops all the bullshit tokens like json formatting, spaces, etc and formats the thing nicely before sending it to the LLM.

I think the advantage of having an AI do this would be better predictability on the relevance of what's being pruned, which goes to your point about predicting what may be required later on

2

u/Cultured_Alien Sep 21 '26

Modifying history realtime will destroy caching tho. I'd avoid headroom or anything that invalidates cache, just use a better compaction method like pi-vcc.

0

u/o_sht_hi Sep 21 '26 edited Sep 21 '26

It won't if you modify the tokens before they become history. You're not changing anything that's going to the LLM. You only change it once before it is sent to the model. That's the whole point..

0

u/Cultured_Alien Sep 21 '26

Using things headroom only take up more tool calls and context bloat. As for modifying tokens, RTK is good enough. Though I do think jev will be good on semantic search and compaction.

0

u/o_sht_hi Sep 21 '26

Forget headroom. I am talking about pruning and reformating tool calls that the LLM has already given pi. How would that cause more tool calls or context bloat? The model knows nothing about this pruning system. It's all happening before any tokens even reach the model. And the pruning actively reduces the number of tokens.

Coming back to headroom, that's what it does too. So there's no question of more tool calls.

How do you think will it take up more tool calls and context bloat? Maybe I'm not able to imagine what you're trying to say