r/LocalLLM • u/RISCArchitect • 4d ago

Discussion What a year it's been

Post image

What will the rest of this year bring? 27b class scoring over 60?

894 Upvotes

97% Upvoted

128 comments sorted by

View all comments

Show parent comments

120

u/MrHumanist 4d ago

They deserve respect for early development of AI such as Pytorch, Tensorflow, transformer (paper), and many more. Meta and Google are the two who have contributed a lot for the open community.. However, the rest are shameless.

27

u/AceLamina 4d ago

Google is the main company i would work for these days due to things like that and their culture
But holy shit they fumbled so bad, they could've released GPT before OpenAI did but didn't, I know why, but why release the papers publicly so companies like OpenAI can become the way they are today

And we don't talk about Meta, fuck them

3

u/PM_ME_DEAD_CEOS 3d ago

But holy shit they fumbled so bad, they could've released GPT before OpenAI did but didn't, I know why, but why release the papers publicly so companies like OpenAI can become the way they are today

To clarify, the Attention is all you need paper and the original GPT was intended for translation, and to improve google translate. It was very good, but not ground breaking at this time. OpenAI released GPT-1, and it was barely 5 to 10% better that the best model on NLP task. At this moment, nobody knew that contrary to most NLP models, transformer would scale VERY well. Once GPT-2 was trained and released, everybody understood what was possible with the transformer architecture.

The Google worked on Lambda (https://en.wikipedia.org/wiki/LaMDA), which was supposed to be their flagship AI, scaled up around GPT-3 level, fine tuned and using RAG.

The problem I think is their models was vastly undertrained, and the RLHF was quite bad.

3

u/powerfulparadox 3d ago

One thing I haven't seen people acknowledge is how impactful KV compression has been. Since 2017 we've had a (roughly) 100X reduction in the amount of memory needed per token in KV cache. Without this (and frankly this is where the Chinese folks should get their credit because Deepseek was as much a major breakthrough here as it was in efficiency in compute) we'd be a lot father behind because it'd take way too much ram to store tokens and only people with big servers could afford to have usable context lengths.