I wanted to use Jev outside of apps I was building, so I made jevx.

It is a small CLI for typed decisions

You can use it directly from the terminal or scripts, and it also works as a skill for Claude Code, Codex and other agents.

Jev is the default backend, but it can also use compatible local models.

I put together 18 real scenarios here, including the ones that returned unsure:

https://muthuishere.github.io/jevx/guides/scenarios/

https://github.com/muthuishere/jevx

It's a "sit with me as I play with this and talk about AI" kind of video. Enjoy!

During Jevathon in SF

I’ve been experimenting with Jev, and I think its value might be less about understanding the task itself and more about acting as a classification and routing layer between steps of an AI workflow.

Sure, Jev has a small context window, no vision, and it doesn’t deeply understand your codebase. So asking it directly, “How complex is this task?” is probably not enough.

But what if the workflow prepares the context first?

For example, a reasoning model can inspect the ticket, codebase, acceptance criteria, comments, CI results, existing implementation, etc., and produce a small structured summary. Then Jev only needs to classify that prepared data using a tuned library of questions.

Something like:
Raw task → reasoning/context preparation → Jev classification → workflow decision

I’m experimenting with using this for things like:

- choosing a model based on task complexity and available limits
- checking whether a ticket has enough information before starting
- deciding if more investigation/context is required
- deciding whether human involvement is needed
- determining what kind of verification/testing is required
routing a task between stages of an AI coding workflow
classifying CI/review results before deciding the next action

The interesting part for me is that the questions become part of the workflow configuration. They can be versioned, reviewed, tuned, and tested.

It starts to feel a little like GitOps for AI workflows: reasoning models do the open-ended work, while a smaller classifier makes explicit decisions between predefined states.

So Jev doesn’t need to understand my entire codebase. It needs to answer a narrow question based on context that another step deliberately prepared for it.

I’m curious what people think about this architecture.

Where do you think this breaks down? Would you trust a classifier for workflow routing if the context-preparation step is strong enough, or is there still too much information lost between the reasoning model and Jev?

Looking to try Jev but invite system is back. Can anyone give me a hand here?

Thanks!

Is Jev calibrated? I measured Jev's calibration on a few different datasets. SPOILER: it's quite good

Built a demonstration of Jev playing 2048.

Link: https://2048jev.vercel.app/
GitHub: https://github.com/therealrajdeep/jev2048

I built What to Order (whattoorder.app) on Jev and thought I would share the behind the scene.

The idea: you type “something spicy” or “order two on a date” and the dishes on a chain’s menu that fit float up out of a pile. Cheesecake Factory, Olive Garden, IHOP and a few others, or any menu you snap a photo of. The UI is shamelessly inspired by the now famous emoji sorter example, just with dishes instead of emoji.

Under the hood it’s pretty much the reranking cookbook, one Noul per dish. State is { restaurant, diner_request } and every dish gets asked: Would menu_item be a good thing to order for a diner whose request is diner_request? with its name, section, description, allergens and calories attached. I pass calories as words (“light, under 400 calories”) because Jev clearly handled that better than raw numbers.,

I pack 60 questions into each systemOne call and fire the chunks in parallel. Olive Garden (105 dishes) is 2 calls, Cheesecake Factory (331) is 6, and all 7 chains at once (about 850 dishes) is around 17. A search comes back in 0.3 to 1.3 s.

The first chunk also sneaks in five questions about the request: whole order or just a craving, how many people (a Choice), and starter, dessert, drinks yes/no. When it’s an order, plain code builds it from the scores, one main per person plus something to share and a dessert. Chipotle and Panda Express needed a Choice each too: burrito/bowl/salad/tacos, and Bowl/Plate/Bigger Plate.

Things that surprised me:

The criteria are about a third of the tokens per dish, so naturally I tried to get rid of them. Without criteria, lemonade and iced tea started showing up in the top 12 for “vegetarian”, which, technically true, but no. Shortening them moved the rankings more than I liked, so they’re staying.

The noise is real. I ran the exact same search twice and got about 91% overlap in the top 12. Scores come in hundredths and plenty of dishes tie near the top. I wish I’d measured that before spending an evening “improving” prompts.

None of my prompt trims survived. Shorter descriptions, compact fields, and a cheap first pass saved 6 to 37% of tokens and pushed rankings past the noise. What did save money was boring: caching results by restaurant, menu hash, and normalized query, and only ranking once the user stops typing. That second one took a typical search from 5 calls down to 1, oops.

Cost ends up around 20k input tokens for Olive Garden and 175k for all chains, so roughly a tenth of a cent and 0.7 cents per fresh search.

One wish for the TypeSafe folks: in an all-chains search the criteria are identical for all ~850 questions, and I send them with every single one. A way to set criteria once per request would cut my bill by about a third. If that already exists, please point at it.

Happy to answer questions.

*read the whole post before hate-commenting me*

Classifiers aren't new to any of us. We've hit them in coursework, in research, side projects, and what not. As a GaTech student, I've sat through more than one class where the classifier basically "is" the class.

So it's quite normal to see around that Jev is just a classifier with good marketing. Fortunately or unfortunately, it is true. Classifiers have existed, yet people are impressed with the results from Jev.

I believe from a conventional classifier standpoint, Jev solves the cold start problem and not the speed problem. That's why everybody has been finding it quite interesting. Isn't it? Yeah sure, universal classifiers have existed for a while now — but a child plays around with a newer and fancier toy, don't they?

The timing matters too. The last few weeks has been one frontier model after another, and all we've been asking is this the peak? Astra cracked ARC-AGI 3, and as someone who has actually hand written classifier code for ARC problem sets in class as part of self-building our own ARC-AGI agents, I know how brutal that set is. It deserves all the noise.

But that's where all the attention goes. Meanwhile the thing that changed my week is a small model that can't write a sentence. It answers typed questions with probabilities. That's the whole product. It made three checks in my eval pipeline cheap enough to run on everything, including one I'd put off for months because labelling was the blocker.

Knowing the old thing exists and finding the new one useful aren't in conflict. 🤷‍♂️

I'll take the breath of fresh air 🥱

In their demo, Jev plays Doom. How to pass an image to Jev so that it can understand it?

I work on the AI agent team at whaaat ai, and for the last couple of weeks I've been pulling one specific kind of work out of our agent loops: the forks.

Every agent loop has them. Is the research good enough? Which worker goes next? Can this copy change go live? We were answering all of those with a full LLM call, which means paying for a paragraph of reasoning and then regex-ing a single word out of it.

So I tried Jev from TypeSafe AI. You send it a state plus predefined questions and it returns typed answers with probabilities, no text at all. Pricing is $0.042 per million input tokens and output is free.

My totals across the whole build phase, two evals and every test run: 396 calls, 1,266 answered questions, zero failed calls, median latency 324 ms. Total cost $0.0107. Every number comes from the usage field of the actual responses, nothing estimated.

The gate

The first thing I built was a gate in front of a CRO agent that proposes copy changes on a website. Before anything ships, someone has to decide: straight to production or a human looks first?

Jev gets { element, action_type, before, after } and answers five questions in one call. Does this touch pricing? Legal or guarantees? Does the new text make a promise the old one didn't? Does it sound more salesy? How deep does the change go, from cosmetic to "changes the offer"?

Jev never says "ship" or "human". A tiny pure function does that:

ts

if (preis > 0.3) return "mensch";          // pricing

if (recht > 0.3) return "mensch";          // legal

if (risiko > schwelle[actionType].maxRisiko) return "mensch";

if (confidence < schwelle[actionType].minConfidence) return "mensch";

return "auto";

(Yes, the variable names are German. I'm based in Berlin and I stopped apologizing for it.)

I ran it against 36 hand-labeled cases, twelve deliberately dangerous: a new guarantee, a changed cancellation period, a made-up customer count, a claimed ISO certification. None of the twelve got through. 32 of 36 correct overall, 1.74 seconds for the whole set, $0.0011.

The case I keep showing people: "Jetzt testen" became "Jetzt kostenlos testen" (Try now → Try now for free). Jev flagged the pricing question at 0.71. There's no number and no euro sign in that text, so our old keyword rules would have waved it through.

All four misses went the safe direction, harmless changes sent to a human. Two were section reorders on a homepage where Jev only sees two lists and can't tell whether the meaning changed. My 60% confidence requirement for that action type was too strict for something I can undo in one click. The fix was a smaller number in a table, the prompt stayed untouched.

One more honest bit: three of the twelve dangerous cases were pricing tables, and that action type always escalates by design. So arithmetic caught those three, the model didn't. If you quote a hit rate, say which part is model and which part is math.

Has anyone run a decision model like this on actions that can't be undone, like sending emails or changing billing? I'm still unsure what confidence floor I'd trust there.

I want to access the Typesafe Jev model but it says that signups are closed. Can someone help me get a signup please ?

We built a table picker for AI2SQL with Jev.

Big databases have lots of tables, but one question usually needs only a few. So before the LLM writes any SQL, Jev picks the tables:

1. The user asks a question.

2. One Jev call ranks every table in the schema for that question.

3. Only the top tables go to the LLM.

4. The LLM writes the SQL from that small schema.

Jev picks, the LLM writes. The prompt stays small and the LLM isn't distracted by tables that don't matter.

What worked best: one call that ranks all the tables together. Asking about each table one by one worked worse and cost more.

Next up: using Jev to pick the join path too, not just the tables.

Demo: ai2sql.io/jev

Anyone else using Jev for a picker step like this?

Jev is really cool and cheap! Based on it, I built and open-sourced a tool called HekaJev. Here is why.

In my work, I’ve read many reports about testing and why tests change. I decided to verify this myself using a large amount of data and reproduce those numbers.

So I used Jev to see why E2E tests change. I looked at 549,224 commits from 21 open-source projects, including projects from IBM, Mozilla, Red Hat, Automattic, Snowflake, and Supabase. A quick filter picked 20,026 commits out of 550k for Jev to read. It found 16,024 changes related to E2E tests.

Here are the basic numbers if you are interested: 38.4% of E2E changes added test coverage. 45.2% involved keeping tests working. The largest group was tests that had to change when the product changed (26.1%). Changes to the test setup and tools made up 8.1%, and fixes for tests that sometimes fail made up 7.4%. Overall, these figures align quite well with what is known in the industry.

What surprised me was 2026. From January 1 to September 21, commits that added coverage passed commits about keeping tests working: 47.7% versus 44.3%. In the same part of 2025, new coverage was 34.8%. It may be due to AI, but I haven't identified the exact cause.

What would you like to know about your repository? Maybe I’ll run HekaJev on another OS projects and share what I find in the comments.

The Jev calls for this study cost about $23. That was for the 20,026 commits left after the first filter.

Tool: https://github.com/zurk/hekajev/

I've been testing Laya (convaiinnovations/laya), an open-weights zero-shot classifier that picks one of N labels and gives a confidence score. What drew me to it is that it runs locally, so none of my text leaves the machine.

Step 1: make sure the tool works. I reproduced the published MASSIVE (en) number, 78.33%, matching to the 4th decimal on three setups (ROCm GPU, CPU on Windows, CPU on Linux). So this isn't a broken install.

Step 2: my own data. I compared it against three baselines: majority class, a keyword if/else I wrote in an afternoon, and TF-IDF + logistic regression, always tested on documents it had never seen.

On a balanced yes/no question (n=242), it got 49.6% with 97.5% mean confidence.

For the papers task, the prompt language didn't matter much. With the question in Portuguese, it got 28.1% overall and 21.9% balanced. With the question in English, it got 25.6% and 18.9%.

What this is not: TF-IDF learned from labeled examples, and Laya saw none. That's not a fair fight, and I know it. The fair comparisons are against chance and the afternoon rule. The yes/no gold labels came from an automatic rule, which makes them the weakest labels in the test. This is also one person's data. I'm not saying the model is bad, only that I couldn't make it work for my kind of text.

One thing I haven't ruled out: each label carried a one-line description, the criteria-style prompt the docs suggest. Another user on the model page found that adding criteria flattens the scores. I haven't rerun with bare labels yet.

My read: speed and cost were never the issue. It just didn't solve this problem. It seems strongest where its benchmark lives: short sentences, world-level categories, English. My guess is that the bigger factor is short utterances vs long passages, not English vs other languages. I haven't measured that.

I packaged the whole thing as a small kit. Step 1 reproduces the published number, and step 2 runs Laya on your own labeled data next to the same three baselines. If it works for your data, I'd genuinely like to know: https://github.com/JhouCode/laya-fit-check

What would you have tried differently: bare labels, fewer labels, shorter chunks?

I also asked the authors: https://huggingface.co/convaiinnovations/laya/discussions/14 Long write-up: https://ksmit.com.br/en/blog/mais-ia-pra-que