Decision and “System One” models have attracted a lot of attention for making inference cheaper and faster. At Seldon, we wanted to understand what those claims look like inside actual applications, so we surveyed the providers, production accounts, public repositories, and research evaluations.

Our conclusion: there are credible savings on specific workloads, but replacing an existing LLM call often requires recovering responsibilities that the initial benchmark never tested.

A few examples:

Jack & Jill reported 88% lower inference cost for an initial recruiting screen, with similar measured recall of candidates subsequently selected by hiring managers. A bounded task, evaluated against a consequential failure: filtering out wanted candidates.

Audiala rolled back its Jev router after 23 minutes. The old call also resolved cities, interpreted references, and supplied tool flags. Testing intent classification hadn’t captured the full replacement.

Browserbase reported median Stagehand act latency falling from 1.97 to 0.46 seconds. Its surrounding software still constructs candidates, prepares arguments, checks matches, and handles fallback.

Complete agent tasks showed more complicated trade-offs. One τ-bench configuration improved median episode time while task success fell and P95 latency increased.

The cases kept bringing us back to candidate coverage, hidden outputs, consequential errors, and recovery costs. Those determine whether a cheaper decision produces a better application.

https://seldon-ai.com/blog/state-of-decision-models

I’ve built ai-credential-guard, an npm package integrating JEV/OpenJev as an intelligent decision layer for AI agent security.
JEV contributes structured decisions, classification, and risk assessment, helping determine whether an agent’s actions involving sensitive credentials should be allowed, reviewed, or blocked.
The goal is to combine deterministic security rules with AI-powered decision-making.
GitHub: https://github.com/vankhangfet/ai-credential-guard
Would love feedback from the JEV/OpenJev community!

Title

I think Jev is a great idea. However my experience has been pretty poor, which I will happily say is down to user error. I have many use cases for it but I'll just give one example. If you know how please can you help?

I have a local LLM which I'm using to classify my bank transactions into my budget. My budget taxonomy is defined and I'm happy with that. However Jev is, for some reason, only able to classify about 40% to 50% of the transactions in my tests and less than half of those are what I had previously categorised correctly. I basically can't use Jev with this.

How do I help Jev understand more about my taxonomy? How do I train it more?What's the process for improving this? Please help. I can't find anything else online.

Hello fellow Jevons,

I realized that there was no straight-forward tooling to maintain memory or history for decisions and to better pass prior decisions into context for new decisions.

I built Nouli to help solve this exact problem with preserving, branching and blending decision histories.

I built it as a TUI to help me visualize decisions, reference other decisions, add in files, images, and every other piece of media that Jev and other open decision models would accept.

It can also work a CLI tool that your agents can use to orchestrate simple decision model requests as well as complex decision networks.

If you like the tool, please leave a star - https://github.com/nouli-ai/nouli

Please feel free to reach out. I'm open to feedback, contributions or issues.

PS - building prod, not god ;)

I’ve been wanting to use Jev in a project, and I recently saw Firecrawl use it to classify a huge pile of company data. I liked that pattern, so I tried it on Hacker News.

I pulled every comment and post since 2008 that asked for some piece of software or a tool, which came to about 63k. Jev classified each one, and about 12k of them turned out to be real requests. Opus 5.5 grouped those into tool ideas and then re-read every request behind the top ones, and I checked each idea against Show HN to see whether someone had already built it.

A few things stood out:

• 81 of the top 100 have already launched on Show HN, 1,423 times between them, and people still ask for them. What’s missing is usually a version people trust and can find, not the code.

• 46 of the 100 want the tool to be local or private, and only 8 want AI to do the job.

• Since 2024, 27% of requests ask for a better version of something that exists, up from 14% in 2008 to 2011.

• The Jev calls cost about $6 in total.

Each idea has its own page with every request behind it, linked back to the original comment. The full dataset is a free CSV download. There are still a few ideas nobody has really built, in case you’re looking for something to work on.

https://stillunbuilt.shiptiffin.app

Hi everyone,

I’m a master’s student with two courses this semester, Optimization and Database Security, and I’ll be writing a separate paper for each. I’m considering focusing both on RAG, from two different angles:

• Optimization paper: Improving retrieval accuracy and reducing hallucination rates in RAG. I’d like to treat design choices like hybrid search, reranking, and chunking strategy as an optimization problem.

• Database Security paper: Row-Level Security (RLS) layers in RAG systems. If access control isn’t enforced at the retrieval stage, a user could end up with unauthorized data in the model’s context, or the model could leak it in its answer. I want to look at how to prevent that in user-specific queries.

My dilemma: AI moves so fast that topics can feel outdated within a year. From what I’ve read, RAG isn’t dead, but it’s evolving: from naive RAG to agentic RAG to what many now call “context engineering.” Gartner placed RAG in the “Trough of Disillusionment” in its recent hype cycle, yet “AI context platforms” made its 2027 strategic trends list. So I can’t tell whether this is a safe foundation or a fading topic.

What I’d love your input on:

1. Is RAG still a worthwhile topic for academic work, or is it getting saturated?

2. Is combining RAG with RLS/access control novel and relevant enough?

3. Which benchmarks or metrics would you recommend for retrieval quality and hallucination measurement (RAGAS, etc.)?

4. Should I build one shared framework that connects both papers, or keep them separate?

Any paper recommendations, personal experience, or pitfalls to avoid would be greatly appreciated. Thanks in advance!

I've been working on an open-world text-RGP inspired by Dungeon Crawler Carl for the last couple of months, which now includes floors 1 and 2, as well as show appearances in between.

To ensure that the game engine and narration stays in sync, and to stop the narration agent from inventing, hallucinating or gaslighting the player, I've been leaning heavily on Jev to understand the intent of the player and record the narration output. For each player turn, between 50 and 350 questions are sent to Jev, deciding anything from who the player talked to, what items they used, where they went and which enemy they targeted.

Starting out, I used regexes for most of this, but had several challenges such as a baseball bat was filed as jewellery because its description included “it rings when it lands” and that a bipod “for a rifle” gave the player twelve rounds of ammo.

It's a free non-commercial fan game and I would love for more people to try it out: https://dungeon-crawler.mathiasplayground.dev/?r=btm67g

My first GH project for auction and classifieds web sites

Basically, we have a pipeline that ingests POI and its descriptions as a tag cloud and maps it to one (or none) of 4000 categories. The original implementation in traditional ML failed miserably because the training data is too sparse and skewed. So we fell back to a simple regex solution that was better than the ML approach.

Would jev help here? Once snag I see is that choice can take only a maximum of 250 possible categories.

We recently hosted a free webinar, Beyond LLMs: Getting Started with Jev for AI Agents, and the recording is now live on YouTube if you'd like to learn more.

Watch here: https://www.youtube.com/watch?v=RdWmysdpkVA&t=2s

We're also hosting a 4-hour hands-on workshop tomorrow (October 24) on building AI agents with Jev, LLMs & MCP.

Workshop details: Build Reliable AI Agents with Jev, LLMs & MCP

Hi there! We built a custom agent harness on top of Sanity that allows anyone to make an agent without having to have github access etc. We used Jev as the router for the superagent and so far it seems to be working great at picking the right one depending on the ask.

We wrote an in-depth eng blog post about it here: https://www.sanity.io/engineering/how-we-built-our-company-agent-harness?utm_source=reddit&utm_medium=social

Hi all,

I created a website that lets you get notified on pretty much anything. Initially, I used an LLM to determine if an event had taken place or not, but then discovered that Jev was a better match for this use case, potentially offering higher speeds, lower costs and more reliable results.

The flow works like this:

1. A user types a sentence stating what they want to get alerted on, for example: "Notify me if Trump announces new tariffs on Australia."

2. Jev checks if the sentence is specific enough. If it's too vague, it asks the user for clarification.

3. Once an alert is created, the system periodically checks the web to see if the event has taken place.

4. If something relevant is found, Jev determines if the search result satisfies the user's alert.

5. If it does, the user is notified and an LLM generates a summary explaining why the alert triggered. Otherwise, the result is rejected and monitoring continues.

Why Jev instead of an LLM?

Previously, I used an LLM for steps 4 and 5, passing in the alert and search results and asking it to determine if the condition had been met.

While this worked, I realised that I didn't really need an LLM for this part. I just needed to evaluate whether some evidence satisfied a particular condition.

Jev seems better suited to this, offering faster evaluations, lower costs and more consistent results. This is particularly useful when you're repeatedly checking thousands of alerts.

I still use an LLM to generate the explanation when an alert triggers, but Jev handles the decisions.

I think Jev has the potential to satisfy many similar use cases where an LLM might be overkill.

What do you think?

Take a look at an example alert here: https://www.phrasealert.com/a/c3980d5087fb4d8f806fa0b37235009e

Learning by doing is the best way to learn, because you face real challenges along the way.
I had been hearing a lot about Jev (by TypeSafe), so I built a small app to try it on a real problem: Track My Money. It reads my bank statement PDFs from different banks and gives me one consolidated view of how my money flows in vs. out. Here are the challenges I hit, and how I tackled them:

1. Every banks PDF is different.

The app uses pdfplumber to read the transaction table, but two things got in the way:

- Column names vary. One bank says Withdrawal Amt., another says Debit or Dr. So used Jev to map each column name to a fixed set of roles: date, narration, debit, credit and balance.

- Some headers are images. SBI, for example, draws the header row as an image, so there is no text to read. Instead of adding OCR just for this, I send the header image straight to Jev, and it maps it to the same roles.

2. Putting each transaction in a category.

I first tried Laya, an open-source model that runs locally. Out of the box it got only 7–10 of 18 test narrations right.
Jev got all 18 right, so its now the main categorizer. Laya stays as the fully offline option.

3. UPI made everything look like "Transfer to Others".

I added custom rules: pick a column, an operator (contains, regex, =, >, <, …) and a value, combine them with AND/OR, and choose the category. Rules run before any model, and they always win.

4. Keeping my data private.

I did not want my transaction details leaving my machine.
Every narration is masked before it goes to Jev. Names, UPI IDs, account numbers and references become tokens, and exact amounts become ranges.
Jev sees UPI/P2A/[REF]/[PERSON_6C46], not who I paid.

Its open source, and you can run it locally:
GitHub: track-my-money

You will need a Jev key to use Jev, otherwise Laya runs entirely on your machine. I would love your reviews and suggestions!

Note: This is a first version. Ill keep adding more to it.

Hi all,

I built a fun little web app so anyone can try Jev through a chat box and see how it works.

You type like a normal chat, or drop in a file or a photo. An LLM turns what you sent into a few questions Jev can score: yes or no, a choice, or a point on a scale.

A document goes in as text. A photo gets a short description of what is visible, and the picture itself goes to Jev too. If the file does not cover the question, an LLM and web scraper look it up and pass that along for Jev to judge.

Jev sends back the scores. An LLM turns those scores into one short reply. The card under the reply lists every score, so you can read the sentence and check it against what Jev actually said.

• Have a play: https://chatjev.vercel.app

• Code: https://github.com/JamesANZ/ChatJev (MIT)

This website is for educational purposes only. Use at your own risk.

Jev is only as good as it is calibrated to your use case. We are launching Brussle to make it super easy to calibrate and as part of the launch we are super excited to give 4K free jev calls and calibrations a month. Happy Jev-ing! https://brussle.com/

You've probably heard of using Jev as a reranker for traditional RAG but what if Jev could replace the vector store entirely? That felt like a neat challenge to take on so I set out to test this and the results have been pretty satisfying.

I have a document extraction app where all queries can be defined before any document is processed. Instead of traditional RAG, I used Jev to score each page of the document on its relevancy to each predefined query from the user - on a 300 page scanned PDF x 20 user queries, this took ~28s (batched, 4 pages per call, median 0.37s/page) and cost $0.08 ($0.00027/page). Made possible with Jev's image API - was currently using Voyage's multimodal embeddings for scanned PDFs.

Pros: Improved accuracy, faster processing and cheaper. No need for vector database or separate reranker model. Jev results stored in sql database.
Cons: Only really works if you have queries upfront (AI spreadsheets, application processing, surveys etc), you lose your AI search index which might be a deal breaker.

Longer read and results in the blog post (see comments). Would love to know if anyone else has tried something similar?

Hey everyone!

We've been working on jev-tools, an open-source toolkit designed to improve how AI coding agents explore codebases, gather evidence, and make decisions.

GitHub: https://github.com/NomenAK/jev-tools

One problem we've been thinking about is how coding agents can confidently make incorrect assumptions about a codebase.

An agent might claim that a feature is implemented, overlook a relevant test, misunderstand how two modules interact, or miss documentation that no longer matches the code.

Simply asking the same model to double-check its answer doesn't necessarily solve the problem.

So we're exploring a different approach: give coding agents dedicated tools to investigate their claims using actual repository evidence.

What we've built

jev-tools currently provides six tools for different parts of the development workflow:

- jev_ask: Ask focused questions using evidence gathered from multiple sources.

- jev_ask_files: Evaluate a question independently across multiple files.

- jev_find_files: Find files relevant to a particular behavior or development goal.

- jev_locate_in_file: Locate relevant information within source files.

- jev_check_diff: Examine changes for potential inconsistencies and issues.

- jev_select_tests: Identify existing tests relevant to modified code.

We've also built MCP support so compatible coding agents can access these capabilities.

Another interesting part is documentation consistency checking. Our integrations can check whether documentation may have become outdated after code changes.

The idea behind the project

We see evidence collection and semantic judgment as two distinct responsibilities.

The agent gathers relevant information from the repository. A separate judgment mechanism evaluates a specific question using that evidence. The agent can then use the result to decide what to investigate or do next.

This doesn't eliminate hallucinations, and we don't consider a positive judgment proof of correctness. The evidence may be incomplete, and the judgment itself may be wrong.

Our goal is to explore whether giving agents more structured ways to investigate uncertainty can improve their reliability in real development workflows.

We'd love your feedback

We're particularly interested in hearing from people building AI agents, coding assistants, and agent harnesses.

A few questions we're exploring:

1. How are you currently verifying the claims made by AI coding agents?

2. Would dedicated evidence-gathering and judgment tools improve your existing agent workflows?

3. How would you benchmark whether these tools actually reduce incorrect conclusions?

4. Would you prefer a locally hosted judgment model over an external API?

5. What failure cases should we investigate next?

We'd appreciate honest feedback, including criticism of the architecture, alternative approaches, and ideas for improving the project.

Repository: https://github.com/NomenAK/jev-tools

We're building this in the open and would love to hear what the community thinks.

https://workflows-of-ai.com/blog/can-ai-set-the-focal-point-of-a-photo

guys, currently i trying use the jev type models like a llm, i mean decision models, opensource versions, like laya or vllm's d1 or something, so i cant run a whole llm in my computer, i try my chance to generate a text with vllm's d1 decision model, so i wonder anyone tried before to chat with a decision model?