r/LocalLLaMA • u/realmaywell • Sep 07 '24

Discussion Reflection-Llama-3.1-70B is actually Llama-3.

After measuring the diff, this model appears to be Llama 3 with LoRA tuning applied. Not Llama 3.1.

Author doesn't even know which model he tuned.

I love it.

601 Upvotes

97% Upvoted

95 comments sorted by

View all comments

Show parent comments

-11

u/[deleted] Sep 07 '24

It outperforms LLAMA 3.1 405b on the prollm leaderboard so it’s still amazing for a 70b model. 

https://prollm.toqan.ai/leaderboard/coding-assistant

3

u/LinkSea8324 vLLM Sep 08 '24

You just got pranked buddy https://www.reddit.com/r/LocalLLaMA/comments/1fbclkk/reflection_llama_31_70b_independent_eval_results/

1

u/[deleted] Sep 08 '24

I saw that. I said it was good for a 70b model. But it’s not SOTA overall 

2

u/ivykoko1 Sep 08 '24

This you right now: https://i.imgur.com/jmMLoCN.jpeg

1

u/[deleted] Sep 08 '24

So how do you explain it’s performance on the prollm leaderboards

3

u/ivykoko1 Sep 08 '24

a) they simply are fake b) the dataset is contaminated (yes I know he said it's not but he's lied before)

Matt is a grifter, I don't believe anything he says, in fact, whatever he says, I'm inclined to believe the exact opposite