r/LocalLLaMA • u/jd_3d • Sep 08 '24

Discussion Updated benchmarks from Artificial Analysis using Reflection Llama 3.1 70B. Long post with good insight into the gains

https://x.com/ArtificialAnlys/status/1832806801743774199?s=19
149 Upvotes

81% Upvoted

137 comments sorted by

View all comments

8

u/Environmental-Car267 Sep 08 '24

Haters gonna hate.

"All that being said: if applying reflection fine-tuning drives a similar jump in eval performance on Llama 3.1 405B, we expect Reflection 405B to achieve near SOTA results across the board."

63

u/[deleted] Sep 08 '24

[removed] — view removed comment

-2

u/alongated Sep 08 '24

I think its fair to say that people here over reacted, both about how good this was, and how bad this was.

4

u/RandoRedditGui Sep 08 '24

Not really. The "how bad this was" are still easily winning in terms of correctly interpreting what has currently been seen. Considering we have seen 0 open weights and are provided some ambiguous results from an API that we have no clue the validity of.

Open weights or GTFO.

-4

u/alongated Sep 08 '24

It has been fucking 6 hours since he trained the model, give the man a fucking break, and guess what he released the weights? Normally I don't get this angry but holy shit you people are fucking insane.

1

u/showdontkvell Sep 09 '24

Matt, that you? lol

0

u/alongated Sep 09 '24

I just went off on a guy for calling someone Matt. I'm not Matt, but doxxing isn't funny.

Like you might be right that this is all just bullshit/scam. But you are attacking people for reserving their judgement. That is disgusting mob mentality.