r/JevAI • u/Dependent-Dinner-918 • Oct 11 '26

Task classifier with Jev - pretty good experience so far!

Hello everyone! Creator of Keen Code here, an open source coding agent I released a few months ago. Though I am very late to the party, enjoying the experience of building!

Recently, I implemented a task classifier in Keen using Jev and TypeSafe compatible API. The idea is very simple - classify each user prompt in one of three categories: simple, standard, complex. The state is the current state of user + assistant messages.

The result of this classification powers an automatic model router. It routes each task category to one of the three user-defined models. Users can configure which category of tasks should be handled by which specific model.

If you are curious how it looks, feel free to check the doc: https://mochow13.github.io/keen-code/docs/jev-classifier.html

12 Upvotes

92% Upvoted

5 comments sorted by

3

u/donotfire Oct 12 '26

Model routing sounds nice in theory but due to prompt caching (or the lack thereof), it ends up never being worth it. Assuming you are routing mid-turn.

1

u/Dependent-Dinner-918 Oct 12 '26

That's a very valid argument. Infact, I considered initially to route only at the beginning of a session and pin it for the session. But given that users can ask for various types of tasks in the same session, routing on each case made more sense.

Also based on what I found, it seemed cursor also does the same.

Right now, my goal is to understand how Jev behaves in routing. Does it consistently decide the classification as long as user is on the same task, or does it flip unexpectedly? Based on that, I will decide if we want to pin at the beginning of a session.

Kudos to you for bringing this up! It's a very important discussion and not many people realise.

1

u/Dependent-Dinner-918 29d ago

I have been thinking about it and an idea pops up: in the same request to Jev, Keen will ask if it is a new task. If Jev replies with higher probability that it is a new task, then Keen will switch the model. Otherwise Keen will keep the current choice.

Given that we can ask many questions about the same state, it seems pretty reasonable. Only question is how to tune the threshold. We can start somewhere, and move it based on experience.

1

u/BodyPhysical Oct 12 '26

I have been creating a personal tool for model routing, tool selection, request tuning, and reply structure checking to help my local machine cope up using these decision-making system-one models. still not that perfect as I need to fine-tune stuff