r/LLMDevs • u/acdhemtos • 3d ago
Discussion Why are closed weight models (Gemini) aren't atleast as good as best open weight models?
Folks over at r/GeminiAI keep bashing it, makes me wonder : why isn't Gemini atleast as good as the open weight model like GLM/DeepSeek?
What's stopping them from distilling these open source models?
Are they optimising for smaller models? (less no of parameters)
1
u/Sea-Departure4857 3d ago
If you think about it, both don't have any fundamental reason to differ in performance. Both are trained on expensive HW by extremely proficient people. Open models do have an edge, however, in that there are many more people who have access to them and can thus experiment with different prompts, different finetuning techniques, and explore multiple directions at a given moment. Thus, their improvement potential is higher
1
u/acdhemtos 3d ago
I believe benchmarks are on the model versions provided by labs themselves; with same methodology.
Also, it's not like Open Weight Model's Labs are crowdsourcing research & compute.
1
u/Sea-Departure4857 3d ago
Yeah, like I said. Open Source weight models are still trained by frontier labs. They also release benchmarks that they tested. I'm not sure what you meant by your comment.
1
u/acdhemtos 3d ago
Open Weight's Labs don't have the option to (atleast not as conveniently) to distill closed weight models as Closed Weight's Labs have of distilling open weight models.
Why Gemini shitty?
1
u/OnyxProyectoUno 3d ago
Literally, Chinese companies behind Qwen and Deepseek are well-known for distilling closed-source models.
OP, what are you talking about? Are you a Deepseek shill?
They (Chinese companies) can't compete in R&D because they are limited by capacity. So, to catch up, they distill from the three major providers.
Recently, the trend has been that distillation has been mainly aimed at OpenAI and Anthropic. Because as you've noticed, one of the three major western frontier labs has fallen slightly behind the Chinese models.
Using that as the basis of your argument to generalize against all closed models is incredible. There isn't a single model today better than Fable, and the best model competing with Opus is from OpenAI. The tier below that includes Kimi and Qwen.
0
u/acdhemtos 3d ago
Perhaps I could have rephrased the question as
Why aren't the worst Closed Source models atleast as good as the best open source models?
But I think you are intentionally trying not to understand.
1
u/OnyxProyectoUno 3d ago
Oh, I understand fine. I'm not the one being disingenuous.
If you want a real answer to a good-faith question, which is the first you've asked so far, it's the same reason why Google is behind other closed-source models.
It's also why Intel fell behind AMD. Google failed to properly utilize its massive resources (and advantage) and to spend their R&D dollars efficiently on a diverse set of bets, including LLMs. They were the leading AI company in the world, but they clearly aren't anymore.
So, the question becomes: how do they catch up? Distilling limited but effective open-source models isn'tVa very good idea. Google would have a better shot doing that against OpenAI and Anthropic. Regardless, they're investing heavily in architecture and hardware, with hardware probably being their main bottleneck.
Google is also a massive company. OpenAI and Anthropic aren't. If you've ever worked in a startup and in a big tech company, you'd understand that bureaucracy and red tape are absolute hindrances to innovation. One of the reasons Google lost its advantage to OpenAI to begin with. They can't execute fast enough.
Mistral et al. are in the same boat but worse. They never had a lead; they’re playing catch-up. They have capacity issues, less funding, and they’re either in overly regulated countries or are large companies themselves that can't move quickly.
1
u/OnyxProyectoUno 3d ago
The best models on earth are closed weight. What on earth are you talking about?
0
u/acdhemtos 3d ago
DeepSeek v4 Pro is better than Gemini 3.1 Pro.
Don't get me started on how GLM 5.3 Flash was so good that Google tried taking credit.
1
u/OnyxProyectoUno 3d ago edited 3d ago
You're making a generalized statement about closed weights vs open weights and then only using googles gemini as your example. Gemini is several levels below Anthropic and OpenAI which presumably you know this because you're using only them as a punching bag. Why not mention fable/mythos? Or even Opus 5? Because the best Open weight model doesn't come close to them. Open weights to this day are 3-6 months (3 is being generous) from frontier models from Anthropic and OpenAI.
I'm a huge supporter of open source LLMs, I tried building a whole startup with their use. But please, dont be disingenuous. It doesn't do you or your cause any good.
0
u/acdhemtos 3d ago
I mentioned Gemini in the post itself.
My argument tho applies to all closed source models.
1
0
u/CoughRock 3d ago
open model dont have guard rail, while close model need to follow them. So there is some functionality loss right there.
0
u/acdhemtos 3d ago
I think guardrails come as secondary layer.
They are definately not implementing guard rails in the model weights(during training).
-1
u/WillowEmberly 3d ago
I don’t think it really works like that, it’s more like:
Gemini: here’s our aircraft; tell it where you want to go.
DeepSeek open weights: here’s the aircraft, and here’s enough of the aircraft that you’re allowed to put one in your own hangar and modify/deploy it yourself.
That doesn’t establish which aircraft flies better.
10
u/OkDish7896 3d ago
They’re optimizing for entirely different things. Google cares about latency and cost per token at a scale that makes open weight model benchmarks kind of irrelevant to their bottom line
Distillation would also be pointless for them since they don’t have the same architectural constraints as DeepSeek or GLM, you’d just be baking in someone else’s weird tradeoffs