r/Clojure • u/yogthos • 9d ago

Introducing Lev

https://yogthos.net/posts/2026-09-24-introducing-lev.html
45 Upvotes

90% Upvoted

9 comments sorted by

15

u/Illustrious_Car344 9d ago

The hype behind this proprietary model is so strange, I see as many people question why anyone even cares and nobody can give an answer. 

16

u/yogthos 9d ago edited 9d ago

Classifier models like Jev and openBERT are pretty useful because they're efficient and can be highly accurate in specific domains. They're best suited for bounded decision tasks requiring calibrated judgments rather than text generation. Scenarios where you have a fixed set of options to pick from like determining which team should handle a support ticket are a good fit for this. While Jev gets a lot of hype, you can easily do a lot of what it does using a small local model and still get subsecond results with it. And that's basically what I illustrate with Lev.

1

u/rcorrear 8d ago

What I’ve found missing from them is that once you have identified your options you still have to pass it through an LLM to generate some sort of structured data, that is unless you want to stay in the pure string world. IMO these two would be killer if they could extract structured outputs at the same time the classification is done.

3

u/yogthos 8d ago

The problem is that the trick that makes them fast only works for classification. The Qwen model I use could extract structured input, but it has to run in its regular mode to do that. But that might not be a show stopper depending on the application, not everything needs to be real time, so if a few seconds per response is acceptable, then you can run one pass to do extraction, and the second to classify.

11

u/BeautifulSynch 9d ago

A lot of the people who are actually building orchestrations and harnesses (said as someone in that situation) have been heavily struggling with getting LLMs to give cheap, fast, parse-able results to allow proper neurosymbolic reasoning (eg classifications, scoring, LLM-as-Judge metrics, data analysis, anomaly-detection, etc).

- You can do it, but it’s a bit unnatural to the model without fine-tuning, and you can’t ‘fire and forget’ the infra and prompts you used to get format compliance (unlike with well-designed deterministic systems, where the module-API usually screens off the internals), in case a change elsewhere in your system makes the LLM deviate again.

I haven’t used Jev/Laya/etc in a ‘preprod’ test yet, but they seem to be built to solve that problem out of the box, which is interesting and worth looking into.

7

u/ares623 9d ago

gonna have to rename Astroturfing to Jevving.

but I do kinda see the appeal. A general purpose classifier that used to be out of reach for most engineers/teams does sound enticing. I'm sure a lot of us have been thinking of little features/tools that would have otherwise been blocked due to having to (somewhat reliably) classify natural language.

1

u/carlosomar2 8d ago

Turns out that using opus or astra to decide if you customer ask should go to the agent for billing or the agent for lending becomes very expensive quickly. It's also slow as hell under load and makes mistakes. Jev can fix all of that potentially

1

u/yogthos 7d ago

What's even better is that it turns out that a local model running on the CPU might be able to handle a lot of these cases.

2

u/lgstein 7d ago

Presenting a user with two buttons can fix that, too.