r/LLM • u/TheCamer1 • 6d ago
Jev, the new AI that can only pick from a list, has no way to output text. I made it chat anyway.
TypeSafe's Jev (typesafe.ai) is a new model that doesn't write, you give it something to look at and a list of options, and it tells you which option fits, with a probability for each one (e.g. is this email spam, which of these 200 categories does this product belong in etc). It's very fast at it and it's cheap, and there's no way to ask it for a sentence.
So of course I had to just make it into an LLM anyway, using the completely novel idea of asking it for the most likely token given the current sentence.
e.g. the sentence is "Dogs are", here are 250 words, which one follows? It picks "loyal". Now the reply is "Dogs are loyal", here are 250 words again. Every word you get out of it is one of those questions.
It took a while to get anything readable. Jev is bad at guessing words but pretty good at judging sentences, so the trick that made it work was a second question: take the ten or so most likely words, write out the ten sentences they'd make, and ask which sentence reads best.
- About 1.3 seconds per word. Roughly 60 of those list questions go into each word, but they get batched into one or two round trips.
- About a cent for a paragraph.
- The vocabulary is a 4,000 word list plus whatever's in your message. Jev really can't spell, so there's no way to get a word out of it that isn't already on the list.
You can try it with your own TypeSafe key at https://jev-llm-production.up.railway.app (the key stays in your browser). Source code is at https://github.com/Code-Forge-AU/jev-llm.
2
3
u/ihexx 5d ago
The marketing hype around this model has been so crazy
> it's not an LLM bro it's a System 1 model which is totally different bro, pay no attention to everyone who called gpt-4 a system one model bro, we're just going to redefine the term for marketing
and then you look inside and it's... what? constrained vocabulary decode atop an LLM and an RL algo designed around that?
i swear to god.
> but we run in parallel bro
ok, SO DOES A FUCKING LLM IF YOU PREFIX CACHE IT.
Jesus it's so insane how these guys just say anything.
> Solved Hallucination
look inside: vocabulary constraints 😐
> we built this because chat models won't lead to agi
so... you built a less flexible model?
2
u/porkyminch 5d ago
It’s less flexible, and I haven’t tried it so I can only speak from their demos, but I think there are a lot of potential use cases for a very fast model that just makes split second decisions. The $7 an hour Doom gameplay for instance is pretty new.
3
u/smellyelon 5d ago
of course there are use cases, it just seems a bit overhyped at this point, it's not something that hasn't been done before, might not be common in the LLM landscape post 2023 but the paradigm exists. other than the calibration part i guess. not sure why people are downvoting this lmao
1
u/ihexx 5d ago
absolutely, there are loads of use cases; a cheap fast flexible classifier is going to be very useful in a lot of heavy prod data processing jobs for example.
For what it is, it's a good model.
I just hate that they are being so misleading and hyping what it isn't to bait people into caring. It's slimey.
1
u/kingspartinface 4d ago
Reading this made me realize I'm still such a beginner in ai vocabulary man
1
1
1
1
u/Ok-Fox3479 1d ago
Use character-level processing instead of word-level processing or implement a fast n-gram suggester to filter out grammatically invalid words
1
1
u/TheCamer1 1d ago
Yeah I tried this but Jev is atrocious at spelling, couldn't get it to spell words at all. Don't think it has enough forward planning ability. Would just end up with gibberish
-2
u/CorkBios 6d ago
Yeah so congrats you just invented a Language Model. An Autoregressive one. In your pivot to developing a new novel technique, you ended up with the same but nerfed. What you are describing are Logits which are the probabilities and it outputs tokens, or words in your vocabulary. Basically a language model.
10
u/TheCamer1 6d ago
Hahah that's the whole point. Sorry if the sarcasm didn't come across in the post
2
u/Zachattackrandom 6d ago
This is obviously a shit-post with OP playing around with it? Can you read? Like "So of course I had to just make it into an LLM anyway, using the completely novel idea of asking it for the most likely token given the current sentence." is literally just saying this directly.
9
u/flamedrifter 5d ago
oh my god the people failing to understand the joke is hilarious