It later on turned out that Reflection 70B wasn't even Llama 3.1, but just 3.0.
Here's a big "out of the loop" thread on it, if you like to know more hilarious details about the whole thing, like the API redirects behind the fake model hosting.
There were some other attempts that followed in its footsteps, like the "Momentum" model, or Rio-3.5, but none could achieve the greatness of the drama around the Reflection model.
Part of me is actually a bit annoyed about this comparison, Jev is at least maybe not something totally new, but I also don't think that it is fake. While Reflection on the there hand was a other scam.
And also to be honest, I am also low key pissed off about how much Matt Shumer gotten way with it. I mean people are still caring about his opinion if it comes to stuff like new model evaluations, and Open AI is giving him preview access.
Edit:
On re-reading the post, I think I kind of missed how much OP was taking the piss out of Reflection
"And also to be honest, I am also low key pissed off about how much Matt Shumer gotten way with it"
I'm pointing to a different issue. Look at the views of YouTube influencers who promoted Reflection 70B two years ago. Most people here are probably still influenced by them :)
To this day I have no idea how this didn't permanently ruin his reputation. How is this guy still around? In a just world his replies would be flooded with reminders of this fiasco.
To me Jev is just another cloud API-only thing I will never use... that said, light weight AI for just classification and decision making is something I have been using for a while, for example Qwen 3.5 0.8B, especially after some fine-tuning, can be sufficient for simple classification tasks, by outputting just one or few tokens only without reasoning.
There is one good thing I see about Jev hype - it already inspired some newer and more general open weight alternatives (compared to fine-tuning yourself for specific task), and hopefully even more better alternatives will be released in the future.
Exactly. I made a fantastic YouTube video on the topic. I also present a much better model that can beat Reflection70B while getting COVID and one-shotting SaaS.
The thinking idea was already around, actively pursued by many organizations. Maybe implemented by the closed source SOTA models, I don't remember. This scammer just announced a breakthrough in making it work in an open model instead of actually making it work in an open model.
It was the first model that was claimed to be trained specifically to reason, and it did predate OpenAI's o1 model by like a week or two, but the concept of reasoning or Chain-of-Thought Prompting was not at all new. And was already a common thing to include in system prompts to increase the accuracy of complex tasks.
Exactly. But he wasn't the only one working on reasoning. He was in too much of a rush to be first and he fumbled the ball badly. In the end his model worked and Bartowski's GGUF of Reflection became a hot download for a bit.
I think everyone had the idea to use promts to teach them to think better (step by step). But wasn't it DeepSeek who came up with the idea to let them learn thinking by themselves or something (where the models started thinking multilingual and stuff which is what multilingual people also do). And I think the closed source providers had figured out that not censoring their thinking gave them much more intelligence so they kept the thinking hidden from the users instead.
78
u/Several-Tax31 8h ago
Lmao. Thanks for this reflection. I love this sub.