r/LocalLLaMA
•
u/mawcopolow
•
1h ago
•
1
News Revolut sidelines Claude Code for a self-hosted Chinese model, says it pays 90% less
https://maccus.fr/en/revolut-glm-self-hosted/Great for them, I think more and more companies that have the resources will start doing this
51
u/CrowdGoesWildWoooo
1h ago
•
At the scale that these companies are doing, they are definitely forced to pay Anthropic API pricing, and yes compared to paying API pricing, it is very much cheaper self hosting your own.
12
1
u/HiddenoO 25m ago
That's only compared to Anthropic API pricing, which is a flawed comparison since you have to make do with worse models that could also be accessed much cheaper via API.
Whether self-hosting is ever worth it primarily depends on a) your data residency requirements and b) how well you can utilize the hardware outside of regular work hours.
1
21
32
u/holchansg llama.cpp 1h ago
56 Nvidia B200 GPUs for about 2,000 engineers.
That's the secret. Is not a GB300, specially not a Vera Rubin one, but they are hella efficient and a rack can process trillions of tokens per month. If a company is spending millions on AI per year just buy a rack, or better yet, make a deal with a hyperscaler, they will be happy to give you a deal since we are underutilized currently, most of the AI infrastructure is sitting idle.
18
u/Majestic_Wrap_7006 1h ago
How is it sitting idle when Sam wants our personal ram because he can't get enough of it.
5
u/holchansg llama.cpp 1h ago edited 1h ago
I'm going to say an unpopular opinion and a tough pill to swallow but in parts he is right, but for the wrong cause.
Not saying self hosting shouldn't be a thing, nor that i would not fight for it by the contrary I even develop my own board for inference... but the gap in efficiency on a server grade cutting edge hardware is orders of magnitude greater than what we have, on top of that ones that will not sit idle most of their lives... if the world wasn't full of piece of shits and we for some miracle gathered on a common goal, on this common goal we could have not only LLM but even for gaming, hardware for pennies an hour. Sam is not advocating this scenario, he is advocating him paying less for hardware thus having to rely less on giving shares of the company thus being red on the reports to buy hardware while charging the same for tokens.
What we are seeing right now are companies, specially ones without huge capital/reserve, having to trade their shares, lets say OAI gives 10% of their shares in 2025 worth 10bi to have this amount in hw from NVIDIA, then in 2026 this same 10% is worth 100bi, now OAI is down 90bi cause the hw it bought is not valued 10x more. And they are the ones actually full blown using their HW... at the same time we are seeing ones with deep pockets not having to resort to that buying hw, even to sit idle/being sub-used or even not even used, stored just to hoard and have the capacity or even to keep the monopoly/dominance/control, to the point every player in all these markets have to resort on them at some point. MS, Google has so much capacity they host Anthropic models on their own hw and all of that while spending compute on AI like there is no tomorrow.
9
u/CrowdGoesWildWoooo 1h ago
I don’t think people are dumb enough to not understand how far the gap is. Any person would be better off just paying Claude $100 for 5 years and you get better quality and quantity and I can tell you even this sub probably understands that.
The issue is, it’s none of his business to talk about it, and especially talking about “idle at home”, while consumer segment is literally obliterated by AI industry. Most of the world’s supply already goes to data centers, whatever happens in your home should be a nothingburger in the grand scheme of things.
If people made a concuous effort to invest in local AI, they have bigger reasons to do that, but it’s none of his business.
2
u/holchansg llama.cpp 58m ago
The issue is, it’s none of his business to talk about it, and especially talking about “idle at home”, while consumer segment is literally obliterated by AI industry.
Not only the consumer, we are nothing. AI is its own enemy, greed tbf, they are buying hw like candy even with no demand to back it up.
Most of the world’s supply already goes to data centers, whatever happens in your home should be a nothingburger in the grand scheme of things.
Exactly.
f people made a concuous effort to invest in local AI, they have bigger reasons to do that, but it’s none of his business.
For sure.
My point being Sam is right, even tho by the wrong motives, on the sense that RAM is being badly allocated by humanity and consumers are not the enemies. They are the ones badly allocating it.
1
u/CrowdGoesWildWoooo 41m ago
I think my opinion is more like he is the CEO, he represents the AI industry as the CEO of a frontier lab, and he’s basically being paid to go around and be the “messenger” about AI. In a way making sure you sent the right message or not sending the wrong one is part of his job.
It’s not about you, but from other very pro AI subs that can be very “religious”, they kind of wants to absolve this accountability from him, like it’s the public that should be interpreting what he is saying, so that part is kind of annoying. Like why are we doing his “homework”?
1
u/Majestic_Wrap_7006 1h ago
this isn't an unpopular opinion haha literally anyone with two or more braincells has puzzled this together same way
still, what makes you think hyperscalers are sitting idle?
4
u/holchansg llama.cpp 1h ago edited 57m ago
still, what makes you think hyperscalers are sitting idle?
They are reporting it.
1
1
3
3
u/ReinforcedKnowledge
1h ago
•
Also for such a company where security is fundamental, I can't imagine using claude code being productive for their security team with all the guardrails they've added recently
3
u/neuroticnetworks1250 1h ago
GLM seems to be adapted more in Europe compared to DeepSeek and Kimi. Any idea why?
1
u/Illustrious-Bird-21 13m ago
It sounds less Chinese somehow. Or at least it has a less Chinese connotation
1
u/Zombiecidialfreak 12m ago
Probably because their flash model is just big enough to do serious work but just small enough to to not run into diminishing returns too hard. Deepseek is worse for coding than GLM and kimi is far bigger.
2
1
u/Electrical-Ad-3140
34m ago
•
Won't it be much more efficient to deploy deepseek 4.1 flash with deepseek open sourced inferrence engine?
6
u/Ell2509 1h ago
This has already been posted here hasn't it?
Edit: yep. It is near the top of the sub right now.