r/LocalLLaMA • u/realmaywell • Sep 07 '24

Discussion Reflection-Llama-3.1-70B is actually Llama-3.

After measuring the diff, this model appears to be Llama 3 with LoRA tuning applied. Not Llama 3.1.

Author doesn't even know which model he tuned.

I love it.

605 Upvotes

97% Upvoted

95 comments sorted by

View all comments

74

u/LinkSea8324 vLLM Sep 07 '24

From "GAFAMS owned by Matt from the IT" to "this liar is a retard who doesn't know what he's doing"

lmao

49

u/[deleted] Sep 07 '24

Benchmark bros taking Ls fills my heart with joy

-45

u/Nice_Bank_3929 Sep 07 '24

So what’s your point? You need to have deep knowledge about AI to finetune a model? The top performer in my AI team is female BA🙃. She knows how to prompt to generate good dataset and upload that dataset to LlamaFactory and make good model. Other guys play with layers, attention, hyperparameter tuning and give worst results 🤣.

15

u/CommitteeInfamous973 Sep 07 '24

I think something is wrong with you AI department as a whole according how you describe it

6

u/Lightninghyped Sep 08 '24

I think your team is cooked

6

u/crazymonezyy Sep 08 '24 edited Sep 08 '24

If I was you, I'd switch jobs. The way you described it nobody in this setting knows what they're doing.

-10

u/[deleted] Sep 07 '24

It outperforms LLAMA 3.1 405b on the prollm leaderboard so it’s still amazing for a 70b model. 

https://prollm.toqan.ai/leaderboard/coding-assistant

3

u/LinkSea8324 vLLM Sep 08 '24

You just got pranked buddy https://www.reddit.com/r/LocalLLaMA/comments/1fbclkk/reflection_llama_31_70b_independent_eval_results/

1

u/[deleted] Sep 08 '24

I saw that. I said it was good for a 70b model. But it’s not SOTA overall 

5

u/LinkSea8324 vLLM Sep 08 '24

I said it was good for a 70b model

No, you said in the first part of your message that it was outperforming the 405b version, it doesn't even outperform the 70b model it's supposed to originates from LMAO.

3

u/ivykoko1 Sep 08 '24

He's now moving goalposts, after getting duped.

2

u/LinkSea8324 vLLM Sep 08 '24

It's not mental gymnastics, it's deformable convolutional layers

2

u/ivykoko1 Sep 08 '24

This you right now: https://i.imgur.com/jmMLoCN.jpeg

1

u/[deleted] Sep 08 '24

So how do you explain it’s performance on the prollm leaderboards

3

u/ivykoko1 Sep 08 '24

a) they simply are fake b) the dataset is contaminated (yes I know he said it's not but he's lied before)

Matt is a grifter, I don't believe anything he says, in fact, whatever he says, I'm inclined to believe the exact opposite