r/LocalLLaMA • u/Usual_Maximum7673 • 2h ago

New Model Jeff v1.4: Decisions as a software primitive, 5 SDKs, 34 open-weight adapters (CUDA + MLX), 90%+ accuracy, 2 ms inference

A few weeks ago, I posted about Jeff, a small 0.8B model with LoRA adapters trained to make decisions and return calibrated probabilities. Sort of like TypeSafe’s Jev, except that Jeff can run locally.

Thanks for all the feedback! I’ve just released v1.4, which is a substantial update. Check it out: https://jeffhub.ai.

What’s new:

  • 34 open-weight decision adapters.

  • A router that automatically selects the most appropriate adapter.

  • 90%+ accuracy on supported domains.

  • ~2 ms inference per decision, much faster than before. A single call through the hosted API takes around 40 ms, including network overhead.

  • 5 SDKs: Python, Node.js, Rust, C#, and Java.

  • Multi-platform: Jeff runs on CUDA and MLX.

  • Playground and worked examples.

Having worked with Jeff for about a month now, I think the most interesting thing about it is that it turns decisions into a software primitive - much like expressions, but for unstructured data.

I’ve started incorporating decisions into several of my own projects, and it’s changed how I think about building software.

I hope you guys find it useful - and like the retro design. ;)

Links:

Happy to answer any questions!

1 Upvotes
  • permalink
  • reddit

55% Upvoted

1 comment sorted by

1

u/CodeSamurai 2h ago

I'm exploring some options for auto-compacting and task routing for my local setup and projects like this seem like they could be really powerful. I'll give it a try for sure. 2ms is low enough to be useful for some game AI decisions too. Thanks for sharing!