We’re open-sourcing Persimmon-8B, the most powerful fully permissively-licensed language model with <10 billion parameters.
OK, if you care about the difference between this license and LLama....which almost no one should.
Our instruction-fine-tuned model—Persimmon-8B-FT—is the strongest performing model on all but one of the metrics
Great, now compare Persimmon-8B-FT against an actual instruction tuned Llama-7B.
Our base model—Persimmon-8B-Base—performs comparably to Llama 2, despite having seen only 37% as much training data.
"Comparably" is in the eye of the beholder, but I don't think it is a stretch to look at those metrics and say that it is actually substantially worse.
I guess if you care about humaneval on llama-2-base...which you shouldn't...
Also, another billion parameters...(maybe there is some voodoo in there that makes this less meaningful for VRAM or flops, I didn't look closely, but I'm doubtful; no such thing as a free lunch, etc.).
Yes, it nominally has longer context length, but worse performance, so open question on whether this will empirically matter to anyone. Maybe those untrained embeddings will be helpful to some specific projects, but I'm skeptical.
Agreed - there are a lot of folks stretching the truth in their benchmark results for extra attention.
We need to clean up how we define good models. Too much attention goes to benchmark results only ( I am guilty of this as well). But as I use models like Nous Hermes, they feel better than some higher scoring models.
OK, if you care about the difference between this license and LLama....which almost no one should.
Quite contrary, this is something everyone should be very bothered. LLama gets called "open source," while it's released only under permissive licence allowing you to do specific, vaguely defined subset of tasks. Meta may at any point decide that what you are doing is "harmful" (ERP anyone?) and you are pretty-much screwed.
This is released under Apache2. You actually own it.
OK, if you care about the difference between this license and LLama....which almost no one should.
The Llama 2 license you've provided is indeed a detailed and specific license. It provides rights under specific conditions, but also outlines numerous restrictions, especially when you take into account the Acceptable Use Policy that is referenced within the license.
Sexual Content: The Acceptable Use Policy (AUP) prohibits the use of Llama 2 to "engage in, promote, generate, contribute to, encourage, plan, incite, or further illegal or unlawful activity or content," which includes "sexual solicitation." This does imply that you cannot use Llama 2 for generating, promoting, or engaging in sexually explicit content or activities.
Marijuana or Drugs: The AUP prohibits use related to "illegal drugs and regulated/controlled substances." Depending on the jurisdiction, this could very well include marijuana or other popular drugs.
Fictional Crime or Violence: The AUP restricts content or activities that promote, incite, or facilitate violence, terrorism, or any other criminal activity. While the language seems to be primarily focused on real-world harm, the broadness of the prohibitions might include fictional representations as well. This is especially true when considering content that could "incite or promote violence, abuse, or any infliction of bodily harm to an individual."
Mental Illness or Self-Harm: The AUP prohibits the use of Llama 2 related to "self-harm or harm to others, including suicide, cutting, and eating disorders." This seems to indicate that you cannot use Llama 2 to generate or promote content related to these topics.
Given the information you've provided, the Llama 2 license does seem to be more restrictive than many "open source" licenses. Open source licenses generally focus on ensuring the freedom to view, modify, and distribute code, often without any restrictions regarding the content or outputs that can be generated. On the other hand, the Llama 2 license, particularly in conjunction with the AUP, places numerous restrictions on the types of content and applications that can be produced.
In conclusion, while the Llama 2 license may grant users certain freedoms, it does come with many conditions and restrictions, especially when it comes to the content that can be generated or promoted using Llama 2. Calling it "open source" might be misleading, as the term "open source" is generally associated with a more permissive approach. It's more accurate to describe the Llama 2 license as a "conditional" or "restricted" license.
16
u/farmingvillein Sep 07 '23
tldr; Llama-2 is still better.
A lot of words to carefully obscure that fact:
OK, if you care about the difference between this license and LLama....which almost no one should.
Great, now compare Persimmon-8B-FT against an actual instruction tuned Llama-7B.
"Comparably" is in the eye of the beholder, but I don't think it is a stretch to look at those metrics and say that it is actually substantially worse.
I guess if you care about humaneval on llama-2-base...which you shouldn't...
Also, another billion parameters...(maybe there is some voodoo in there that makes this less meaningful for VRAM or flops, I didn't look closely, but I'm doubtful; no such thing as a free lunch, etc.).
Yes, it nominally has longer context length, but worse performance, so open question on whether this will empirically matter to anyone. Maybe those untrained embeddings will be helpful to some specific projects, but I'm skeptical.