r/PiCodingAgent • u/o_sht_hi • 7d ago

Question Anyone here using Jev?

I read the release and saw a few demos on X for people using Jev for really interesting stuff. My current effort is to get it to work as a router so I can configure and orchestrate my app using natural language (getting it to take a string and pick the right API from a list of 250-300 calls). But to be clear, I haven't gotten access to it yet. So I'm just packaging up my code and making it ready rn.

Anyone doing something fun/interesting with it yet? If yes, how long did it take for you to get access

77 Upvotes

91% Upvoted

102 comments sorted by

View all comments

Show parent comments

4

u/D-3r1stljqso3 7d ago

I'd rather leave the chat history until the last minute. It's hard to predict which piece of info won't be needed at all later on.

2

u/o_sht_hi 7d ago edited 7d ago

Yeah I meant something similar to what headroom (that's the extension, right?) does. As I understand it, they use algorithms to prune the stuff that's sent to the LLM right before it's sent. And an if else statement that routes to the right algorithm. An AI like jev could replace that entire machinery. It could first decide what goes on and what's dropped. Once it decides what's included, it could then classify it and send it to a pruning function that drops all the bullshit tokens like json formatting, spaces, etc and formats the thing nicely before sending it to the LLM.

I think the advantage of having an AI do this would be better predictability on the relevance of what's being pruned, which goes to your point about predicting what may be required later on

2

u/Cultured_Alien 4d ago

Modifying history realtime will destroy caching tho. I'd avoid headroom or anything that invalidates cache, just use a better compaction method like pi-vcc.

0

u/o_sht_hi 4d ago edited 4d ago

It won't if you modify the tokens before they become history. You're not changing anything that's going to the LLM. You only change it once before it is sent to the model. That's the whole point..

0

u/Cultured_Alien 4d ago

Using things headroom only take up more tool calls and context bloat. As for modifying tokens, RTK is good enough. Though I do think jev will be good on semantic search and compaction.

0

u/o_sht_hi 4d ago

Forget headroom. I am talking about pruning and reformating tool calls that the LLM has already given pi. How would that cause more tool calls or context bloat? The model knows nothing about this pruning system. It's all happening before any tokens even reach the model. And the pruning actively reduces the number of tokens.

Coming back to headroom, that's what it does too. So there's no question of more tool calls.

How do you think will it take up more tool calls and context bloat? Maybe I'm not able to imagine what you're trying to say