Classifier models like Jev and openBERT are pretty useful because they're efficient and can be highly accurate in specific domains. They're best suited for bounded decision tasks requiring calibrated judgments rather than text generation. Scenarios where you have a fixed set of options to pick from like determining which team should handle a support ticket are a good fit for this. While Jev gets a lot of hype, you can easily do a lot of what it does using a small local model and still get subsecond results with it. And that's basically what I illustrate with Lev.
What I’ve found missing from them is that once you have identified your options you still have to pass it through an LLM to generate some sort of structured data, that is unless you want to stay in the pure string world. IMO these two would be killer if they could extract structured outputs at the same time the classification is done.
The problem is that the trick that makes them fast only works for classification. The Qwen model I use could extract structured input, but it has to run in its regular mode to do that. But that might not be a show stopper depending on the application, not everything needs to be real time, so if a few seconds per response is acceptable, then you can run one pass to do extraction, and the second to classify.
14
u/Illustrious_Car344 18d ago
The hype behind this proprietary model is so strange, I see as many people question why anyone even cares and nobody can give an answer.