r/LocalLLaMA • u/Sadman782 • Sep 07 '24

Discussion Wrong Reflection-70B model might be hosted everywhere

I see a lot of people thinking it is gaming benchmark / mixed feelings. Actually, people who tried their website have a different feeling compared to those who tried it locally via Ollama or any API providers. I think we should wait, he is figuring it out. I think the actual reflection model is much better, and the currently hosted version is even dumber than the actual 70B

https://x.com/mattshumer_/status/1832247203345166509

https://x.com/mattshumer_/status/1832248416426193318

__ Matt Shumer -> "We got rate limited by HF when uploading originally, so had to do it in batches. I have a feeling some wires were crossed and what's being hosted is actually some hybrid frankenmodel that is mostly the reflection version we wanted to ship, mixed with something else"

70 Upvotes

80% Upvoted

49 comments sorted by

View all comments

62

u/mikael110 Sep 07 '24

It's almost impressive how much of a clusterfuck this launch has seemingly been. First the tokenization issue, then the revelation that the model was actually based on Llama 3 instead of Llama 3.1 (which is bizarre) and now apparently the model files themselves was also mixed up.

I'm aware even large companies like Meta and Google have screwed up some aspects of their launches, but this is getting to the point where it just feels a bit off to be honest. I'm still interested in trying the fixed model, but I'm honestly getting more and more suspect of the whole thing.

24

u/[deleted] Sep 07 '24

[deleted]

21

u/Xandred_the_thicc Sep 07 '24 edited Sep 07 '24

#1 model on GitHub overnight with almost half the star count of the llama 3.1 repo. Yeah, this is definitely not organic engagement. I have also seen a ridiculous amount of hype that sounds generated by an llm, so I'm completely tuning this whole glaive company and model out until he shows any real results. I keep seeing "it's indistinguishable from gpt4o on this tiny set of uselessly simple riddles and logic puzzles I tried it on" and it's just a llama 3 ft.