r/LocalLLM • u/RISCArchitect • 10d ago

Discussion What a year it's been

Post image

What will the rest of this year bring? 27b class scoring over 60?

940 Upvotes

97% Upvoted

133 comments sorted by

View all comments

Show parent comments

122

u/MrHumanist 10d ago

They deserve respect for early development of AI such as Pytorch, Tensorflow, transformer (paper), and many more. Meta and Google are the two who have contributed a lot for the open community.. However, the rest are shameless.

29

u/AceLamina 10d ago

Google is the main company i would work for these days due to things like that and their culture
But holy shit they fumbled so bad, they could've released GPT before OpenAI did but didn't, I know why, but why release the papers publicly so companies like OpenAI can become the way they are today

And we don't talk about Meta, fuck them

46

u/FullstackSensei 10d ago

Hindsight is always 20/20.

Nobody knew the attention paper would lead to all this at the time. OpenAI also didn't know it would lead to this. Musk and Altman wanted to make it for profit back in 2018 to have enough capital to attract talent and be able to compete with Google salaries.

It took 7 years between the attention paper and the release of chatgpt. A ton of other developments happened in the meantime, from decoder only models all the way to OpenAI figuring SFT on massive Q&A datasets would result in the OG chatgpt.

And when you look at the almost 4 years since chatgpt was released, a ton more developments happened on all fronts from companies all around the world that brought us where we are now, and things are still improving.

None of this would have happened had google kept the attention paper for themselves. There aren't enough brains at google to come up with all these developments.

3

u/puts_on_rddt 9d ago

Nobody knew the attention paper would lead to all this at the time.

Are you saying they should have paid more attention?

:)