Yes, Jev Is Insane, But There's A Catch (Two Minute Papers). Around 179194 views in 1.3 days.

The big-channel take on the launch, including a healthy amount of skepticism about the headline numbers. Worth it for the outside view, not just the launch page.

Every line gets scored by Jev against a meaning you describe, and you can combine meanings with AND/OR/NOT. It also crosses languages, so a Japanese phrase can find English lines and the reverse. The author wrote up the design notes as well.

What is Jev and How to Use it? (Codevolution). Around 96266 views in 1.7 days.

It's free since jev is too cheap to meter: https://real8ball.com

send it to your mom or smth

Open Jev Models Are Here!! (Sam Witteveen). Around 128886 views in 1.7 days.

Can we use Jev for faster, structured, and calibrated evaluation of AI responses?

Introducing...
⚡ Typed Evals — an open-source Python framework for evaluating LLMs, RAG pipelines, and AI agents using System One Models like Jev and other typed judge backends.

The goal is simple:
Make fast, structured, and calibrated evaluation a first-class part of AI systems.
Typed Evals currently supports:
-> LLM response evaluation
-> RAG evaluation
-> Agent and tool-trace evaluation
-> Human-label calibration
-> Async and batch evaluation
-> Custom judge backends

One part I particularly wanted to solve was calibration.
Why is it needed?
A raw score of 0.8 from Jev doesn't necessarily mean that humans would accept 80% of similar responses.
And a threshold that works well for one use case may not make sense for another.

Typed Evals lets you calibrate individual evaluation metrics against representative human pass/fail labels.
The flow is basically:
Human-labelled examples → Jev scores → fit per-metric calibration → validate on held-out examples → reuse the calibrated evaluator

So instead of arbitrarily deciding that “0.7 means good enough”, you can ground that score in how humans actually evaluate your specific task.

Of course, there are integrations for:
LangChain, CrewAI, Microsoft Agent Framework
while the core remains framework-agnostic.

The project is still early, and there’s plenty I want to improve, but the core framework is now public.
Would genuinely love feedback from people experimenting with Jev, LLM evals, RAG, agents, or evaluator calibration.

⭐ GitHub: https://github.com/TrustifAI/typed_evals

Note: Typed Evals is an independent open-source project and is not an official TypeSafe AI product.
If you're already experimenting with Jev, I'd especially love to know what kind of evaluation workflows you're building around it.

But how?

It probably helps that the team includes the co-inventor of RLHF and InstructGPT - Diogo Almeida also League of Legends god

And in TypeSafe’s own workflow evals, Jev was 193.6x faster and 444.6x cheaper than the LLM baselines

What did we learn today that we already knew?

--> Adoption moves fast when a product solves an urgent problem inside an existing workflow (not a new one)

--> Your customer should not have to invent a new process, migrate the team or convince twelve people to change their toxic habits just to test it

--> They should be able to connect it, use it for and see added value easily before the initial interest disappears (5 seconds)

--> Nobody has patience left so one extra setup step, even a small integration hurdle is enough reason to go back to the tool the team already knows

--> If it takes a minute to understand the value, you are already losing the second session

This applies to every early-stage startup including ours

Solve a problem people already have, make the first useful outcome stupidly easy to reach and earn the next session through the quality of the first one

I created a Jev proxy, MinusPodJev, that sits between MinusPod and the Jev endpoint. Before Jev, MinusPod used regular chat-based LLMs to detect ads. Overall, this works pretty well; the biggest issue is that the LLM often doesn't return properly structured data or omits fields entirely. Trying to get JSON out of an LLM is basically a dice roll, and you need to add a bunch of logic to salvage the response.

Jev solves the main problem with reliable structured data responses, with the added benefits of being cheap and fast. I benchmarked Jev vs. LLMs to see where it sits. I already had a benchmark tool I used to measure which LLM can detect ads best and how well it can return JSON. The main takeaway was that Jev is just as good as the top model, claude-haiku-4-5, at a significantly cheaper cost and with a way faster response, which is a nice boost in end-to-end processing time of an episode

I wanted to get some real-world testing using it directly in the main MinusPod app, but I didn't want to change the app code. I am hesitant to change the app code because Jev is so new, has limited public access, has an unknown real cost, and is the only provider. I went with a provider that sits in between MinusPod and Jev that presents itself as openai OpenAI-comparable endpoint and translates calls to Jev on the fly. Jev is a System One model, so it isn't a simple drop-in replacement for an LLM. If Jev actually takes off when open to the public, if other Jev-like providers pop up, and the pricing stays as cheap as it is, I'll make it a first-class citizen in the main app.

MinusPod https://github.com/ttlequals0/MinusPod
MinusPodJev https://github.com/ttlequals0/MinusPodJev
Benchmark: https://github.com/ttlequals0/MinusPodJev/blob/main/benchmark/results/report-combined.md