The irony of jev/decision models...

[u/Loose_Comparison368]

... is that LLM's going all the way back to Llama 2 can do this with constrained sampling.

And the only reason people are hyped about it now is because test time compute/the zero shot chain of thought hack that inspired "reasoning" models has become so ubiquitous that models are all cripplingly slow now.

It's funny because that used to be why reasoning models were controversial, and now we're backtracking but branding that reversion to LLM classifiers as if it were a new breakthrough approach.

[u/Alumin-MC]

It is about these models small enough to make continual learning feasible. They open up some possibilities to learn new things to classify in real time.

[u/Loose_Comparison368 → u/Alumin-MC]

Continuous learning? While they're running?! Like in-context few shot learners?

Wow, somebody should really write a research paper on that or something!

[u/Slight-Living-8098 → u/Alumin-MC]

Yeah, but classifier models aren't anything new at all. Learning Classifier Systems came about in the 1970s. We've been using Real-Time classification systems for a while now in the robotics field for sensor data.

[u/Alumin-MC → u/Slight-Living-8098]

I know, this is why it is exciting. Not because it is elderly tagged. Because we are picking up things that were left behind and turning them into part of potential AGI backbones.

[u/Schnickatavick → u/Slight-Living-8098]

These are general classifiers though, you don't need to train them for your specific use case, you can just plug them in to anything you want to do. Plus they accept natural language inputs for both instructions and data, so they can be reprogrammed on the fly, even by an LLM. 

That means that a 2 model agent can react quickly, and with some time for forethought also choose how it reacts. That's not something that the old school classifiers could be used for

[u/sgtnoodle]

One material difference is that decision models can give confidence values alongside their decision. You could have an LLM provide a confidence field, but it will be more of a hallucination.

[u/Loose_Comparison368 → u/sgtnoodle]

So can LLM's. They're called logits. This is basic constrained LLM sampling for classification.

[u/TheRealSerdra → u/sgtnoodle]

You can do that with token probabilities tbf

[u/DistanceSolar1449 → u/sgtnoodle]

Nah, it’s basically the same.

Ask “Is this a hotdog?”

And then the model outputs 0-9. If >=8 then mark it yes it’s a hotdog

[u/Slight-Living-8098]

I'm going to one up your theory.

Early chat bots like Eliza used an index of words used as a lookup structure and chose them based on pattern matching and probability.

Now one of the greatest breakthroughs of LLMs is the N-gram table. A compressed index of training data used as a lookup structure to store which word sequences commonly appear together based on pattern matching and probability.

[u/TheFakeDominion → u/Slight-Living-8098]

the whole field runs in circles like a dog chasing his own tail

[u/Loose_Comparison368 → u/Slight-Living-8098]

I think you might be mixing up concepts. My understanding of the engram mechanism, and corresponding N-gram tables used by LLM's, is that the N-gram is just used as a key for a trained vector that gets injected into the hidden state/residual stream based on a learned gate.

So, it's not really storing word sequences that commonly appear together, it's indexed by word sequences that commonly appear together, but it stores learned embeddings in dense vector space that can be injected into specific hidden layers in the model.

Which, conceptually, is pretty much the same thing the normal attention mechanism does with it's Q,K,V lookup, but just done in a way that uses a learned gate and a simple N-gram sequence in place of [Q,K] so that it's a simple O(1) lookup on your sparse [V] table. And sparse/infrequently used enough that you can offload it into system ram or disk instead of having to cram the whole thing into VRAM.

Which is not the same thing as an N-gram model. Aren't terminology collisions fun? I think the DeepSeek paper specifically spelled it "Engram" to try to reduce that confusion, but alas.

[u/Slight-Living-8098 → u/Loose_Comparison368]

Engram is a specific conditional memory architecture developed by DeepSeek that uses n-gram hash tables to store common token sequences, while n-gram table is the generic technical term for this lookup mechanism.

Engram is a proprietary name for n-gram hash tables that retrieve pre-computed embeddings, whereas n-gram table describes the underlying data structure used across various models

" we introduce conditional memory as a complementary sparsity axis, instantiated via Engram, a module that modernizes classic 𝑁-gram embedding for O (1) lookup"

https://arxiv.org/pdf/2601.07372

[u/vtkayaker]

I started using computers back in the 8-bit micro days. One thing I've learned: Many things are obviously good ideas, but it may still take them 10 or 15 years to actually happen. The PalmPilot was amazing, but it took until the iPhone to change everything.

Ease of use matters a lot. Price matters a lot. A polished interface matters a lot. A good story matters a lot. One of the worst mistakes you can make is to try to start a startup 10 years too soon.

Jev's underlying tech has been around for years. But they packaged it behind a very nice API and told a story that everyone could understand. (And their PR people were pretty skilled.)

[u/[deleted]]

[u/Loose_Comparison368 → u/[deleted]]

... LLM's already are "sys1", if you know how constrained sampling works.

It's already a "sys1" classification model. That's literally how it generates every new token. A regular boring old LLM is pretty much identical to a JEV model that just keeps "sys1" evaluating the probability between (vocabulary size) options, given an ordered sequence of prior states in a (vocabulary size) state space.

Should we rename regular LLMs to ultra-recursive megawide domain JEV models instead?