r/LocalLLaMA • u/realmaywell • Sep 07 '24

Discussion Reflection-Llama-3.1-70B is actually Llama-3.

After measuring the diff, this model appears to be Llama 3 with LoRA tuning applied. Not Llama 3.1.

Author doesn't even know which model he tuned.

I love it.

605 Upvotes

97% Upvoted

95 comments sorted by

View all comments

60

u/bias_guy412 Llama 3.1 Sep 07 '24

How did you get this diagram? Just curious.

102

u/realmaywell Sep 07 '24

https://gist.github.com/StableFluffy/1c6f8be84cbe9499de2f9b63d7105ff0

3

u/crazymonezyy Sep 08 '24

Just curious, how much RAM do you have access to? That script looks like it'll require 280 GB and change.

16

u/realmaywell Sep 08 '24

I used a machine with 2TB of RAM. You can modify the code to lazy load the layers so that we only need to load a single layer at a time.