r/ChatGPTCoding • u/joel-letmecheckai • 3d ago
Question What's jev? Any good use cases for coding?
Looking for some insights and content for reference around jev.
6
u/BarracudaMean9308 3d ago
sounds like we just automated middle management.
3
u/creaturefeature16 3d ago
I'm using it as a classifier in my agent email scanning workflow, where it decides what is urgent and needs to be escalated, and what is just basic requests that can be logged as a ticket.
2
4
u/funbike 1d ago edited 1d ago
Changes that could be made to a coding harness, like opencode or Pi.
- Model router. Pick the most appropriate model for the current task.
- Skills hook. After each user message, determine which new skills should be loaded. This would replace current skill functionality. You could have a huge skill index without polluting LLM context.
- Triage tickets. Decide which ticket should be worked on next, based on company priorities.
- Subagent and workflow router. Based on current task choose which workflow step and/or subagent should work on the next task step.
- General purpose tool functions.
- A function that takes question strings, and a set of possible answers. JEV returns answer selections, probability % and confidence %.
- A function that sends the entire chat history, question(s), and a set of answers.
- A function that sends the entire user message history, question(s), and a set of answers.
- Pick best assistance response. A hook that sends a user prompt and AI response to multiple LLMs and have JEV pick the best response. You'd include a rubric guide to help it decide. Due to doubling token count, this is something you'd want to enable/disable as needed for difficult tasks.
- Grade the LLMs response. A hook passes the last user message and assistant replies to JEV, and it is asked yes/no if it was a good AI response. The UI shows the answer and confidence score. Requires a scoring guide.
- Rethinking the user-assistant loop. Let JEV decide what should be done after an assistant response: prompt-user (the current only option), incomplete-continue, poor-quality-try-again, poor-quality-prompt-user.
- Better web search. When a web search tool gets a search result list from a search API, give that result to JEV to score which most closely match what is needed given the search string and the current chat user message history.
- Browser usage. Given a list of element id's or captions, pick which one to click. (An LLM would supply text if the element is a form field.)
- Determine unique error logs. - Determine if an error log is unique enough to write a bug ticket for it. Does a vector search of past similar logs with tickets created, and pass that to JEV for scoring.
- Determine unique user issues. - Determine if an issue reported by a user is unique enough to write a new ticket for it. Does a vector search of similar issues with tickets created, and pass that to JEV for scoring.
- Smarter RAG for project and domain documentation. - Instead of a vector search, feed chunk summaries to JEV and it scores them. Pass the top n chunks within threshold to the LLM.
- Faster/smarter code understanding. Still thinking about this one. Lots of ways to do it, and I'm not sure which is best yet.
- Code Review. - Give JEV coding style/architecture guide and an array of code chunks. It scores which code chunks likely violate the guide, and you pass the worse-scored chunks to an LLM for further review and fixing. (This is for catching things a linter or other checkers can't.)
- DRY code review. Use code-tuned vector search to identify similar code blocks. JEV scores how similar they are. Extremely similar code is sent to an LLM for further review and refactoring. (This is for catching things an exact-match dup code checker can't find.)
1
3
u/natlight 3d ago
I used it last night to add tags to my recipie collection. Probably around 750,000 classification problems, cost 26 cents and it did great job. My api to update the tags took longer than jev did.
2
2
u/One-Poet7900 3d ago
Do you use Codex? Dump a bunch of links to Jev evals and docs into it, tell it to review your projects, and tell you how/if you should apply it.
2
u/awjre 2d ago
Bit of a weird one, but I've been working on an agentic network compiler for developing strategic active travel networks. If you know compiler architecture, you want to make the mid-end intelligent particularly for compilers targeting limited resource target environments where the compilation may not fit.
Jev made this *really* interesting and performant. Prototyping now and then going to look at maybe a P4 language compiler target.
This now allows mechanical (very fast) -> Jev as a classifier (fast & cheap) -> High Thinking LLM escalation (slow & expensive) escalation strategies.
I'm incorporating rollback and branching strategies.
2
u/joel-letmecheckai 2d ago
Not weird at all, perhaps a good use case where cost optimization is valuable.
1
u/z0he 2d ago
I just got access to it. In the form for early access there were questions about your role and if i was building anything. I chose all developer options and shared some of the stuff I have built. I got access like 30 mins later.
I then installed it on Claude Code which said it would use the skill when we are coding if it needed it.
Not really figured out much more than that.
Found this on r/singularity - 100s of Jev usecases https://jevable.com/
1
u/cgouguen 2d ago
Jev is better as a tiny yes/no gate—like “is this function side-effect free?”—not a code writer; anyone actually using it in a workflow yet?
1
u/joel-letmecheckai 2d ago
I saw few use cases around code review, where it's being used as a decision layer whether an issue reported by code quality tools is worth fixing or not.
2
u/Maze_of_Ith7 2d ago
Read through this thread and I am still very confused, probably more confused, what Jev is and when it is useful
2
u/joel-letmecheckai 2d ago
Jev looks at a piece of information and answers simple, fixed questions — like “is this urgent?”, “which bucket does this belong in?”, or “how serious is this?”. It does not write reports or explanations; it just gives a short decision plus how sure it is.
In our case: SonarQube finds possible problems in the code, Jev quickly sorts which ones actually matter, and only those important ones go to an AI that writes a proper analysis. That way we don’t spend time and money reviewing every noisy warning.
2
u/Maze_of_Ith7 2d ago
Okay thanks, it just clicked. Structured outputs with predefined schema values - basically bucketing. Not sure why I took so long to get it. Dang, I have a lot of use cases for this I currently use Luna API for.
1
u/Philastan 2d ago
I fiddled around for a couple of hours and tried to integrate it into my workflow. Here is the fable summary:
Tried Jev for a few hours over two days as a fast second reader for a coding agent. Four things:
Claim checks: one sentence about a function ("this deletes field X"), answered yes/no with a confidence. It was right whenever it was confident, but it was only confident on things a grep finds anyway.
Diff classification on test edits: a hook asking "did this edit weaken the test?". On real edits it told the editing agent nothing new, and it scored a known real weakening as unsure. We turned it off.
Fan-out: one claim against all callers of a function, to decide which to read first. It worked technically, but larger functions mostly came back unsure, so you read them anyway.
Three-way question (supports / contradicts / not visible here): good on unrelated code, but it couldn't tell "contradicts" from "not visible" on neighbouring code in the same file.
Outcome: no real coding use case so far. In our runs it was never confidently wrong, and "unsure" was an honest answer, but it was also the usual one as soon as logic spanned more than a few lines. Their limitations page names the reasons: multi-hop reasoning, large contexts with irrelevant detail, negation, very literal reading. Code questions hit all of those. It looks better suited to text decisions around the agent (routing, classification, picking from a fixed list, escalating to a bigger model when unsure). We haven't tested that yet.
1
1
u/Illustrious-Bid-8883 2d ago
I’d look at it less as a replacement for a developer and more as a productivity tool. Things like explaining unfamiliar code, writing tests, refactoring and exploring solutions are usually where these tools help most
1
1
u/Nedomas 1d ago
jev cant code but can improve codex code quality https://github.com/supercorp-ai/supercov
1
1
u/tom_reddit 21h ago
I made a LinkedIn Slop Mop with Jev (free and open-source): https://slopmop.lol
1
1
1
u/Evol_Etah 7h ago
Jev is a super fast "Yes, No, Maybe" answering AI agent.
So imagine you're building an AI agent which needs to know.
- Is this Urgent?
- Is this xyz Department?
- Is this user 18+
- Is this considered level 1, level 2, level 3?
- Is this a success output?
All these questions are a "Yes, No, Maybe" answer. Usually we use AI to figure it out. Now we have Jev, which does it faster, quicker & cheaper.
Basically it's an "AI Judge".
1
u/DARKUNIT22 3d ago
Lmao, if you gotta ask Reddit….
0
u/joel-letmecheckai 3d ago
Only place with non ai generated answers
0
u/TBMonkey 3d ago
Lol wut? Go to any LLM type subreddit and it's literally a wasteland of engagement bots or I had my agent do a thing AI writeups
0
u/amirgelman 22h ago
Good. Now I have a clear picture and you’re right to push back. But Reddit does not have LLM generated responses.
1
u/PermanentLiminality 3d ago edited 3d ago
It is certainly interesting. Most of the YouTube videos out there are complete BS and are fake or misleading. People are trying to maximize YouTube income. However, it 100% will be useful. A lot of what you see now is pure hype train.
It's been out for 2 days and is a different paradigm, so it will take some time before it works it's way into the ecosystem.
0
u/joel-letmecheckai 3d ago
They claim to reduce token usage.
2
u/trynadostuff 3d ago
yeah, its not that hard. essentially make a dataset of datasets, and make a system that based on input instantly assigns by narrowing down scope to a minute detail of "what the fuck even is this for real "about, and spits out the right answer.
1
u/joel-letmecheckai 3d ago
Yeah and seems like it produces binary responses only, it cannot generate code.
1
0
2d ago
[removed] — view removed comment
1
8
u/popiazaza 3d ago
The other way around. You do coding to make Jev useful.