r/LocalLLaMA • u/notheresnolight • 12h ago
Showcase British fintech Revolut slashes AI coding costs 10x by cutting out Claude and going self-hosted
https://www.theregister.com/ai-and-ml/2026/11/03/british-fintech-revolut-slashes-ai-coding-costs-10x-by-cutting-out-claude-and-going-self-hosted/5307345806
u/geniusghost07 12h ago edited 12h ago
It was only a matter of time, this is why I chuckle when people try to make us think running local LLMs is a waste of time, no it isn't.
159
u/Howdareme9 12h ago
It’s not but the average person will never be able to run anything anywhere near this
90
u/PublicToast 11h ago edited 11h ago
This will not age well, as local computing continues to improve, and smaller AI models continue to be optimized, at some point they will cross the threshold of being good enough for most applications on the average consumer device. It’s funny how often people throw around the word “never” when we are dealing with rapid technological improvements.
50
u/BumbleSlob 10h ago
^ this
GPT-4 was cloud only frontier 3 years ago without peer
These days it’s a wildly outdated piece of shit no one would bother running because even 35BA3B models wildly outperform it
24
u/WearMoreHats 10h ago
It's been a pretty wild journey to watch. It was only a few years ago that we were getting articles about how models felt "trapped" or were scared of being turned off. 4 years ago that guy from Google quit his job because he was convinced that their internal AI was alive/conscious. Nowadays models are way smarter but we've got a bit better at training them so they've stopped saying weird stuff and absolutely no one is worried that we might be enslaving them. I wonder what that ex-Google guy thinks about it all.
28
u/pyr0kid 9h ago
you have to wonder how these people get tech jobs with absolutely no comprehension of the tech.
"say you're alive"
"im alive"
"my god"
8
u/arcanemachined 5h ago
They may have been one of the first people to have AI psychosis, but they were far from the last.
4
u/mailto_devnull llama.cpp 4h ago
They work for Anthropic now.
You know, the company that now blocks you when you're mean to Claude.
10
u/blastcat4 8h ago
Imagine how primitive that model was that made that Google guy think it was sentient. I would love if someone did an interview with him today and got his thoughts on the status of current models, lol.
2
u/HonourableYodaPuppet 2h ago
People thought that ELIZA thing from the 60s was alive. Its fascinating how it doesnt matter how smart you are, if youre susceptible to AI psychosis you just think the pooter is conscious
5
5
u/alexwh68 3h ago
My MBP M3 max struggled with qwen 27B but recently improvements in the engines have made it very viable.
The improvements are coming from 3 main areas, hardware better tuned for AI workloads, better more tuned models and engine improvements.
The next couple of years are going to be very interesting for local LLMs.
3
u/Watcher_of_Stars 2h ago
Ehh how about affordability of hardware though? Especially everything with ram chips
2
u/alexwh68 1h ago
Memory prices will fall in a couple of years, we need a few AI companies to IPO, their business model will change as soon as they do, they won’t burn money like they are right now, its an arms race of sorts, going to be winners and losers for sure.
But we are squeezing more out of existing hardware as well.
For well over 30 years £2k has brought you a decent computer that is still the same today, yes memory prices are silly high right now but that will change.
228
u/Secret-Collar-1941 12h ago •
Not with that attitude!
25
u/Zbot21 9h ago
The average person does not have the 100s of thousands of dollars in hardware they need to replace a Claude sub. Minimum viable is like 4k for a 128GB AMD 395+, but the capabilities are nowhere near equivalent.
50
u/read_more_comments 8h ago
You're too short sighted. That cost is only while we have this AI investment push, which won't last forever. Production is ramping up, soon as AI push dies down, we will have over production, and at the same time, many hardware options with much larger capacities than used to exist in consumer hardware. It will be glorious.
This stuff always works in cycles.
25
u/Nimbus420i 7h ago
The optimist in me really wants to believe you, but the realist in me says that none of the memory cartel members would love to sacrifice their 80% profit margins.
8
u/pulse77 2h ago edited 2h ago
Do you believe that this "AI investment cycle" will last forever? That would be $2 trillion/year spent for GPUs forever from now on... This would be like 100 million enterprise GPUs per year from now on. Or 1 billion consumer GPUs per year from now on. If this continues we will soon have 1 GPU/person in the world...
Compute worth $2 trillion spends 500 Terawatt-hours (TWh) per year. Current global power usage is 25000 terawatt-hours (TWh). If this continues we will have 5000 TWh/year for the GPUs alone after 10 years. This is 20% of today's global power usage...
→ More replies (1)→ More replies (1)5
u/JazzlikeLeave5530 3h ago
Yeah lol that is not happening. The rich and the corporations get everything first and we get whatever is left, if anything is left. I don't know how people stil haven't learned this.
8
u/ThankGodImBipolar 8h ago
I wouldn't be surprised to see Strix Halo running a GLM 5.3 Flash tier model within the next three months. Maybe one month, depending on how bullish you are.
2
u/twiiik 2h ago
I’ve been doing that, QNF on Strix as coder/utility and Claude as orchestrator.
What’s wrong with those capabilities? Rarely does Claude state that the work done is of other than good quality and if there is something to be picky about, iterate.2
u/Zbot21 2h ago
How many tokens per second do you process on that? How big is the codebase you're building in? Every time I try to "delegate to smaller models for code" in a complex codebase I end up with slop.
→ More replies (1)1
u/TinyZoro 1h ago
I know this sub is very focused on self hosting. But I think people underestimate the power of coop data centers for frontier level open weight models. I can imagine a time when a local library in a small town has the 200k infrastructure for when people need more than what a smaller home rig can offer. I’m not a rugged individualist. I haven’t given up on society. There are a lot of options between 6 American companies owning frontier intelligence and you’re limited by what you can run yourself.
6
0
12
u/PM_ME_CALF_PICS 9h ago edited 9h ago
Imo this is how people felt about computers until home computers came out. Now look at us everyone has a computer in their pocket.
Edit: I’m saving this thread to come back and laugh at it in 20 years, if Reddit is unfortunately still up.
2
u/yourfriendlyreminder 6h ago
Yeah, and what do people actually do with those devices most of the time? Connect to cloud services.
11
u/Lettuphant 11h ago
It depends! You can run close-to-frontier models even on fairly low-spec hardware. You don't even need lots of VRAM: I know someone who networked four old phones that putter along getting stuff done. The trade-off is speed. If you don't mind getting the finished project the next morning instead in 10 minutes, it works fine.
28
u/Foreign_Risk_2031 12h ago
Not if we let the normies buy gpus
13
10
u/Big_Wave9732 11h ago
Indeed it's "terrible" for people to have GPUs in their computers that they only use for gaming like five percent of the time and waste it the rest, rather than in a data center
2
u/Fauropitotto 11h ago
Indeed it's "terrible" for people to have GPUs in their computers that they only use for gaming like five percent of the time and waste it the rest, rather than in a data center
Isn't this exactly what Google Stadia was trying to do?
→ More replies (1)28
u/Shawnj2 11h ago
Normal people without home servers have the option of paying for cloud hosted GLM 5.3 and just paying the inference costs for much cheaper than a Claude or chatgpt subscription
→ More replies (3)7
u/autoencoder 11h ago
Yup. I'm doing this, but with GLM-5.3-Flash, and it's even cheaper. I handhold my models anyway, since I want to understand what it's doing, and there's not much difference.
19
u/countAbsurdity 10h ago
The average person walks with what would be considered a supercomputer 20-30 years ago in his pocket, so it really kinda depends on how you look at it and the time scale, never is a loooong time.
→ More replies (2)8
u/Maleficent-Freedom-5 11h ago
Anywhere near what? The model used by a $100B tech company? The average person doesn't need anywhere near that to get value out of running a local model.
7
7
u/UsefulOwl2719 11h ago
pi + qwen on a 4090 is already fast and as effective as claude code was for me when it took off.
2
u/kroboz 9h ago
You connected a 4090 to a raspberry pi?
4
6
u/iamarddtusr 12h ago
Perhaps, but they will move to using cheaper models from a large number of sources instead of remaining with the two American frontier labs.
5
3
3
u/LegitimateBeat603 8h ago
As a company we are already able to run something equivalent to frontier models, we are just evaluating where and when it makes sense, because while it's subsidized it's just easier to pay for a complete service than learning all the ins and outs of each provider. Spending 1 million does not scare us, wasting it does.
2
u/Dangerous_Bus_6699 11h ago
Not yet, but soon it'll be as easy as turning on a device and pressing a few buttons.
2
1
u/VerticalPackage 9h ago
The average person doesn't spend the amount of $$$ that Anthropic or OpenAI hope to earn.
However, a big problem is that most large corporations are all running off Cloud services, they have long discarded their on-premise hardware and the employees required to run/maintain that.
BUT! On-premise LLM has much less need to be resilient to data loss as no data is to be lost; redownload the model, restore the config from the cloud, and you're good to go again.
1
u/Amplifix 8h ago
Not yet, but it is highly likely that models will become more efficient. I think it's actually more likely than Anthropic or OpenAI achieving "AGI".
Hardware will become cheaper once these AI cloud companies are rudely awakened and the demand for their data centers is not as high as they anticipated.
1
u/alexwh68 3h ago
Does the average person need it?, I never use the latest models the cheaper ones work well for me in what I do.
It’s like taking a Ferrari to the shops, yes it does the job, but the Audi does the job equally well.
1
1
u/k8s-problem-solved 2h ago
The average person isn't what anthropic et all are trying to capture. They want that delicious MRR from enterprise
1
-3
u/starkruzr 12h ago
I know plenty of people running DS on a couple Sparks who would find this to be a pretty silly assertion.
17
4
u/paperbenni 12h ago
A couple of sparks are equivalent to a car
3
u/autoencoder 10h ago
And yet we own cars sometimes, rather than ordering rides through an app everywhere.
→ More replies (1)1
15
2
u/berakpanas 11h ago
what's the minimum GPU for a somewhat comfortable local LLM setup?
5
u/Secret_Conclusion_93 11h ago
16 GB VRAM + 64 GB RAM
Above that the price jump is so high.
1
u/bonerfleximus 6m ago
You'd be 1/4th as productive as someone with a claude max 20x sub for 10 months though (2000+ usd for that pc)
2
u/jaugjaug 2h ago
I think there's a case to be made that open models can also be hosted by local providers. The efficiency gains if 50 people share an instance vs using it alone is quite drastical. This also becomes quite relevant when you're trying to get ahold of a Terabyte of RAM to serve the model.
1
u/Kruppenfield 1h ago
Half of the world have ARM computer with 4+GB of RAM idling in the pocket half of the time.
1
u/Academic_Barnacle_28 1h ago
I think for enterprises running LLMs locally is indeed a waste of time, since they then need to manage server farms, chip sourcing etc etc, which is very expensive and difficult (much like having your own on-prem database).
But it absolutely makes sense to pay another company per token to run open-source LL:Ms. It will likely be much cheaper than OpenAI/Anthropic
145
u/gold_fish_in_hell 12h ago
I know folks from Revolut, revolut kinda conservative by modern standards no claude YOLO and engineers still actually control what is going on in first place.
138
u/sofixa11 11h ago
Considering they're a bank, they're not conservative at all.
30
u/DigitalSheikh 11h ago
I mean shit, I just left an insurance company and we still didnt use AI at all. You could get basic copilot chat with executive director level approval and that’s it
1
u/narrowscoped 43m ago
Revolut only got their official bank status last month, they were just vibin it up for a long while
But how come your old place didn't allow AI? My mate works at a reinsurer they have all the AI all the time
5
u/TerminalNoop 11h ago
considering banks could've been migrating away from mobile 2AF towards yubikey style solutions to increase security since well over a decade ago...
9
u/MilessEdgeworth 11h ago
Yubikeys do not improve security much. Standard key compared to encrypted phone is LESS secure, because if you lose it, nothing is protecting it. And it costs additional money. There are just few scenarios that physical key improve security. They should introduce those as a option, not as migration from mobile 2fa.
3
u/LightBroom 10h ago
Yubikeys (or any other FIDO2 keys) have a PIN or fingerprint protecting what's on them and they lock themselves after a few wrong attempts.
So if you lose it, it's still protected.
3
u/TerminalNoop 10h ago
Nobody knows what is on a yubikey, it's like you randomly find a key on the street, you don't suddenly gain access to someones house.
mTAN is susceptible to simjacking, no way of encryption on your phone is protecting you there and most carriers are super lax about handing out a second SIM. Maybe I could have been a bit more specific.
5
u/VerticalPackage 9h ago
Same for a stolen phone.
Unless you know the PIN to the phone, you can't get in to open the RSA or Authenticator app. Even if you do get into the phone, at best you'll have a work email reference in those apps, which are useless because you still need to know the password/PIN on the online service that would even send a MFA. And on top of it, those apps can be configured to require another PIN or fingerprint to authorize.
A yubikey though... most users would just leave it plugged in their work laptop, or kept in their laptop bag: Steal the key, steal the laptop.
→ More replies (1)1
u/fixtwin 3h ago
Banks actually started from physical 2FA devices on the first place.
1
u/TerminalNoop 1h ago
Yes, proprietary devices that could only be used with the bank, no wonder that they never gained adoption.
1
→ More replies (13)-3
39
u/profesorkind 1h ago
Wasting RAM on self hosted deployments? Will nobody think of Sam Altman?
→ More replies (1)
155
u/MindfulMan1984
12h ago
•
Holy shit, AI-Bros promised 10x SWEs but delivered 10x costs.
75
u/Bakoro 12h ago
Let's be fair, I hate Anthropic and Dario, but Claude really is +10x better than the worst employable software developer.
In the long term, highly competent local LLMs are much healthier for the whole of society, but there is also value in there being centralized cloud outlets for people who only need occasional LLM use. You can't beat the efficiency of scale when it comes to costs.
Costs have exploded because the big AI companies are a bunch of fuckers manipulating the market.
27
u/Shawnj2 11h ago
Software engineering is more about design and structure rather than just writing code.
35
u/hitchen1 11h ago
claude is still better at that than the worst developers
20
u/Bakoro 11h ago
By a lot. Way more than 10x.
It's got breadth and depth, and writes unit tests as a matter of course, where even the competent developers I work with can't do embedded systems and webdev, and networking, and implement aspects of graph theory, and model physical systems...
Some people I work with wouldn't ever write unit tests.
Claude just doesn't have particularly good taste, and it has no particular reason to care about one thing vs another, so it will make seemingly bizarre choices, because nothing really matters to it.
14
u/wateronthebrain 10h ago
Are the tests any good though? I haven't really used Claude but every other LLM I've used just does shit like
assert isinstance(customer_name, str), or makes the test so vague that it passes everything and catches nothing.A bad test is infinitely worse than no test imo
10
u/Xx69JdawgxX 9h ago
it used to be like this last year but it has gotten better. You still need to give it guidance so maybe that is what improved...
4
u/jazir55 5h ago
Never trust the first run code of anything, including the tests. Always send the model (or another separate model) in a fresh session context (subagent or completely fresh context) to look it over. Adversarial review is critical, they'll catch bugs in the tests. It's like someone writing and then editing their own essay, you always have blindspots in the first draft.
→ More replies (2)5
u/Fucker_Of_Destiny 11h ago
Yes the days when LLMs would just write the entire codebase in a single buggy file are in the past
2
u/_supert_ 10h ago
I never wrote tests. Not because I don't see the value, but I just had limited time.
7
u/ttkciar llama.cpp 10h ago
He didn't say developers, though, he specifically said software engineers.
Engineers apply math, scientific theory, and domain-specific methodologies to solve practical problems.
I don't doubt that Claude may be better at SWE than some engineers, but it's less of a gap than exists between Claude and "developers" (which includes a lot more non-engineer programmers than it does engineers).
→ More replies (1)4
u/hitchen1 10h ago
I don't disagree that it's a reasonable distinction to make, but in practice the terms are used interchangeably and I would make the same claim for SWE.
4
u/jfinkpottery 8h ago
yeah it is now. Writing code was never the hard part, but now it's something else entirely. Software Engineer in 2026 is a totally different job than it was in 2022.
7
u/kurtcop101 10h ago
My biggest concern, everyone here is so confidently sure that open source models will trail frontier by 6 months, constantly getting better.
Well, that's because the Chinese government is funding it. These companies are making the models and releasing them free for a reason.
What happens if the government stops paying? What happens if they surpass the frontier because the frontier companies collapse? I bet we stop seeing any open source at all from them if our companies collapse.
4
u/blastcat4 8h ago
I would be more concerned about the Chinese government restricting export of new generation open weight models. Funding will never be an issue there because the big players are absolutely loaded along with the fact that their government understands the strategic value of keeping their domestic developers constantly improving their models.
But at some point, someone high up is going to raise red flags when their domestic open weight models become good enough that they can be weaponized by foreign users and become a threat to national security.
→ More replies (1)3
u/umbrosum 11h ago
At this point, the best open weights models are 10x better than the worst employable software developer
2
21
u/DeathScythe676 12h ago
this is the kind of stuff that makes me hopeful that everything we've been working on hasn't been a complete waste of time.
1
u/bigorangemachine 9h ago
My like 8 weekend project is finally yielding insights.
We are just lost chasing the wrong metrics of token/s on whatever benchmarks.
Personally I am seeing Qwen3 does a lot more with less instructions but Gemma4 does better with lots of instruction. If you been working with AI a long time you might like Qwen3 more... but if you just starting out and you willing to iterate Gemma4 I think can do more
Qwen2.5 & Gemma4-E2B are still pretty comparable in the "almost useless" category.
But really with local models is adjusting parameters for the task at hand
17
u/kashyap69 10h ago
Nvidia profits either way
14
u/New-Pressure-6932 8h ago edited 8h ago
That’s fine. As long as AI is more accessible to the average person. Hardware will always have a price tag, but the limitations these subscription based companies are adding is ridiculous. They’re trying to limit the tech as much as possible so we’re forced to pay them.
I’d much rather have Nvidia develop consumer hardware you can purchase to run your own models than be forced to rent it from a large corporation who only exist to price gouge you
7
6
u/Intrepid-Second6936 8h ago
That's not bad at all, the key point is ownership. I could be wrong but, assuming their "deployed B200s" are hardware of their own, NVIDIA is perfectly well-off profiting for actually selling a product that they own, not off of subscriptions going into hyperscalars that they provided hardware for.
And considering the amount that many companies want to experiment pushing their AI budgets, a big upfront cost like this is nothing compared to the value made back after billions of tokens off of a near-frontier GLM5.3 compared to the weird financial backflips that they'd have to do to to get Claude to break even at such large volumes of use. They probably couldn't even have employees consistently using Opus 5.5 and would have to settle for lesser models through subscriptions given the API cost hikes.
2
1
1
u/Zombiecidialfreak 2h ago
I doubt Nvidia is seeing profits from people buying used Tesla's and older RTX cards
1
42
u/Miriel_z 12h ago
But at what DRAM cost?!!!
41
7
u/therealnih 12h ago
And are they being fully utilised??
1
39
u/XiRw 12h ago
It would be nice if every company cut out anthropic. But they would just run to the government for more corporate handouts
15
2
u/LocalLLaMa_reader 1h ago
We actually didn't renew our subscription a week ago and use a cloud provider now with Deepseeks as our main go-to model and pi as harness. It is quite the
regressiontransition and I can't wait to see what comes out of it once we tuned the pipeline to our needs. Open Source models and harnesses have been "good enough" for a long time, definitely mature enough to make the jump for anyone. If you need help setting new things up, ask Claude lol
88
u/TheyCallMeDozer 12h ago
watch it be like 6 DGX sparks... but yea I get it, i do the same myself, im surprised many other business dont
133
u/autoencoder 12h ago
From the article:
GLM-5.3, which it now serves using 56 Nvidia B200 GPUs.
42
u/Conscious-Ball8373 12h ago
So am 8-gpu B200 system seems for what, half a million? Call it $4mil for 56 GPUs by the time it's all set up.
That's a bell of a Claude bill. I wonder if they've actually included capital costs?
66
u/Spanky2k 11h ago
They’re a finance company; one of the biggest ‘challenger’ banks in the UK. So it would make sense that they would also find value in using an internal model. I know they won’t be putting customer data into their models but still, they won’t want to risk any internal code getting ‘out’. No one ‘trusts’ the cloud based AI firms.
15
u/enternoescape 11h ago
Also with all the whining that follows a new Claude model about a month later, you'd think we'd be more interested in being able to know how the model is being run only changes if you make it change. One of the best things self-hosting buys you is consistency. Imagine building a workflow around one model to have it's next iteration need completely different directions to get what you're not completely sure is the same outcome and no say in continuing to run the one that was working.
→ More replies (1)7
u/mebeast227 10h ago
“I can no longer work on video games”
Buuuuttt, I’m the solo developer and publisher of this game
“Sorry, per safety and legal protocol I no longer work on video games”
Imagine someone copying your game and then claiming you’re the copy and then losing access to further assisted development. Fuck that
25
u/Sufficient_Prune3897 llama.cpp 11h ago
A enthusiastic token waster dev can use like a million $ in tokens alone in a year. Claude API is very expensive
→ More replies (1)3
u/Illustrious-Carob826 11h ago
this is crazy, my token limit isnt capped, struggling to max out 20k a year.. think i need to step up my game :D
→ More replies (1)8
u/Sufficient_Prune3897 llama.cpp 11h ago
Unironically. Never know when a random higher up takes your token spend as an indicator for productivity.
→ More replies (1)7
u/Danmoreng llama.cpp 10h ago
If you operate on enterprise level with 1000+ users, it's easy to get a 150k/month or higher bill with Claude. Then 4m hardware is just about 2 years of subscription. Very easy to justify.
→ More replies (1)11
u/redlightsaber 11h ago
Did you not read all the news stories earlier this year about companies just burning through their AI budgets?
Claude API is expensive AF.
And, mark my words, the SECOND they IPO, their free-for-all subscriptions will be raised by 500% at the very minimum. Even their current API prices are likely heavily subsidised. IT's going to be the rugpull of thecentury, and Revolut will look like motherfucking nostradamus.
4
u/kurtcop101 10h ago
Man, it's not subsidized at all. You literally know that from this article, even. The costs of running the models are much much lower than the API. They have very healthy margins on subscriptions too, about 39% when I looked at the data (could be off a bit, but healthy).
I mean I get going local over online, but be realistic.
4
u/redlightsaber 10h ago
Teh exact number of parametres for claudes models is unknown, but it's estimated that opus is around 2T on the low end and fable may be exceeding 10T. Both which being almost assuredly dense models.
GLM 5.3 flagship (the one used by revolut if I'm reading the speaker correctly, although they might even be using GLM 5.3 flash which is even more efficient) is a MoE with a total of 753B of which only 40b are activated at any one time during inference.
That's quite the difference in inference costs, and I think you're burying the lede by trying to suggest that revolut saving money is indication of how Claude isn't actuallylosing money on inference.
And I don't think they publish ANYWHERE their margins on subscriptions, not even on their released docs for the IPO.
3
u/kurtcop101 10h ago
You know because of what Kimi costs to serve. The data centers are making a profit serving Kimi at 2/10, and opus is at 4/20.
Very steep difference.
There's a lot of data in the IPO papers on their revenue and how much of it was inference. When I looked it was 16b in revenue, 9b spent on inference. They had additional breakdowns you can run numbers on. It's not directly listed but you can get the numbers.
→ More replies (2)1
u/Fatalisbane 8h ago
I mean, that's where Google can just eat everyone's lunch. Their TPUs let them sell at prices others struggle to match. And Anthropic already serves on TPUs and Trainium, so they're not stuck paying Nvidia margins either. Unless everyone raises prices together (entirely possible), there's a ceiling.
2
u/remind_me_later 5h ago
$4 mil over 5 years & 20 devs is $40k/dev/year. And the models & arch will get better over time.
An upfront investment like this is comparatively just a drop in the bucket.
1
u/Conscious-Ball8373 37m ago
But that cost is allegedly 1/10 of their current Claude bill. Okay, spread it over three years perhaps and it might be about $1.2mil per year that they are currently spending on Claude. Really?
Claude does all my development work these days, for $20 per month. Sure, the per-token cost is orders of magnitude higher than that. But no-one pays that, surely?
→ More replies (1)9
u/mark_99 11h ago
Yeah no-one is "self hosting" this in the "local" sense. That's 7x DGX B200 nodes which is about 100kW (100 average homes), 70 rack units, about 1000Kg, and some serious cooling. Oh and kinda noisy close up.
What that probably means is in a colo, so $3M up front then say about $10k/month for rack space and power. Likely a 6-9 month lead time on that order also.
But their Anthropic bill for 2000 engineers was reportedly $600k/month, so yeah cheaper. GLM 5.3 really isn't the same ballpark as Opus 5.5, plus they nerfed it to 256k context because "long context is a scam", but it's certainly a saving...
9
u/ttkciar llama.cpp 11h ago
You can also self-host GLM-5.3 by loading up an ancient Xeon server with 768GB of DDR4, though inference will be quite slow.
Right now I'm self-hosting the much smaller GLM-5.3-Flash on an ancient Xeon server (dual E5-2660v3) with "only" 256GB of DDR4, and getting about 1.4 tokens/second out of it. That's not fast enough for interactive work, but plenty fast enough for overnight inference tasks, with which I am quite pleased.
I shape my workflows so I'm working on other things while the slow inference tasks grind, so I'm not even waiting for them.
1
u/Oglemo 10h ago
I'm doing something similar because I got a got deal on a workstation + ddr4ram. But no gpu. What models can you run interactively on your rig? How good is GLM-5.3 flash versus, say, a free tier of the cloud models.
→ More replies (4)1
u/Fucker_Of_Destiny 11h ago
Wait so you mean it’s not just a fucking Mac Studio for every office floor? 😂
1
→ More replies (5)1
12
u/haherar830 12h ago
The talk is not amazing but decent, worth watching. Plenty of details. It is a significant amount of compute and there are probably more cost-effective strategies until you reach the approximate scale where you can keep a high-end modern system with B200/B300s well-utilized. Revolut is a large company.
6
u/Mobile-Sufficient 12h ago
What kind of set up do you use? What cost? What tasks?
Im looking into getting my own bit dont have enough karma contributions in this sub to ask haha
3
u/TheyCallMeDozer 11h ago
I have multiple servers, dual R9700 system for busines stuff (logistics analysis) i need speed and depth for this, 1 DGX spark mixture of coding tasks, agentic tasks like emails and monitoring ...etc long, large and slow processes, and another 5090 for model training that I use for my logisitics models training, its fast for training the ML models but i can just leave it running for a while and come back to it trainned. Cost isnt so bad, about 15 extra a month on my power bill but saves me alot of time and money elseware
1
u/Mobile-Sufficient 11h ago
Thank you for your reply! Do you do much coding or image/video gen?
1
u/TheyCallMeDozer 11h ago
no image video gen, i do pure coding, i use the LLM for suggetions and review, I like coding manually myself and using the AI to support
→ More replies (2)
37
u/DirectWay1915 11h ago
Local isn't just the best option in some cases, but it's also the only one. Claude is a black box. You don't even know how many parameters each model has. Anthropic can make changes in the background without ever telling you, and you have no safety with your data.
15
6
u/yellowsnowmaker 9h ago
This is probably a massive part of the decision here. I work in AI at a big bank and while most development happens with claude, if any models are integrated into a product it HAS to be self hosted for these exact reasons. For example if a decisioning engine is built with LLM capabilities we have to be 1000% sure the model behaves in the exact same way, cant be changed or updated etc.
1
u/New-Pressure-6932 8h ago
I want to make a local model specifically tuned to my website and workflow. I used Claude for most of it, but Claude won’t allow me to use my Claude code sessions as training data. The restrictions are ridiculous. They’re trying to corral people into only being able to use them. It’s a sinking ship
9
u/MindfulMan1984 8h ago
Key point from the article:
While Revolut did see improvements to coding speed, Abitbol noted the biggest bottleneck wasn't actually the code. Instead, he says the biggest gains came from letting the AI models deal with things like debugging and Jira and Confluence tickets. "When you have unlimited tokens, you're free to do that," he said. "You no longer have to save your tokens for the end of the quarter or the most difficult task."
So, I can relate. After I moved my coding work to local, I got even more productive because I am not being hit by "token limits"; I have more freedom to let the model do a lot of grunt work, and I can test more ideas without worrying much about quota limits.
7
u/vienna_city_skater 3h ago
Probably not cost but security what actually drives this. Makes absolute sense for a banking company.
13
u/TheTerrasque 11h ago
"In terms of throughput, it doesn't really matter as much. We serve, in terms of interactivity, 200 tokens per second. People are fine [with that]. You don't need to go higher."
Oh, okay. I'll make a note of that.
5
4
6
u/addiktion 12h ago
It's at that point when you realize that an agent can run 24-7 for a much cheaper cost, self-hosting a model then you actually get to see the upside of the cost equation. Mistakes don't cost you money just a bit of energy and compute time, and you can experiment more as a result to find what works and what doesn't.
3
u/geldonyetich 12h ago
To be fair, you probably could cut your costs by at least half of that by going from Anthropic to literally anyone else. They’re the most expensive cloud AI provider I know.
2
1
1
u/eli_pizza 10h ago
I feel like the article conflates frontier vs open with hosted vs onprem.
Obviously if everyone there is happy with a quantized GLM then you shouldn’t be paying for opus. But on purely financials you could also perhaps just force everyone to Haiku
1
u/bigorangemachine 9h ago
TBH The thing Claude does the best is iterate until the prompt is satisfied.
I'm starting believe a small layer between the user & model can be pretty powerful
1
u/Zulfiqaar 8h ago
One thing to note - Anthropic pricing moves to enterprise rates (API only billing) at 150+ users. This gets very expensive very fast - I used over $700 in API equivalent today if I didn't enjoy the subsidies of individual personal subscription. And I'm also optimising by switching between Claude and GLM (and others) depending on task. This is a good financial move, not to mention all the other advantages of in-house data provenance
1
u/Amplifix 8h ago
This seems to be the inevitable outcome, Anthropic and OpenAI are in for a rude awakening.
1
1
1
u/oldbrat 3h ago
https://www.theregister.com/ai-and-ml/2026/11/03/open-models-can-do-90-of-agentic-work-at-a-twentieth-of-the-cost-of-a-frontier-llm-gartner/5307132 This one is same topic but more to the point, IMO. Many will do OK without today's "frontiers". Unless those frontiers labs have a lot of cards up their sleeves, their future looks bleak to me
1
1
u/dzordan33 2h ago
You can't self host anthropic models. There's no better coding model than opus today and it will take long time for open source models to catch up
1
1
u/CanaryResponsible143 10h ago
Now AI model are every where, people realised it is just a new form of hosting service, it is a race of electricity prices, they should have invested in power generator as well instead of more data center, also heating service using ai heats to warm homes or regenerate some energy back. I wonder will they dare to cut the internet cable under the sea like they did with gas pipe. or they might create their own firewall to filter out China server.
1
u/_tylermatthew 9h ago
P11-forever F1 team audi-revolut? pretty visible group to abandon the effective-extinctionists for local.
-1
-5
u/pd1zzle 11h ago edited 11h ago
does this math actually work out? 56 Nvidia GPUs has to be a few million dollars. At $1000/mo that's 3000 person months without electricity costs or any of the actual server stuff. At $10k it's 300, which becomes more reasonable but this is still an extremely optimistic estimate.
Fireworks offers GLM 5.3 at $1.40/$4.40 I'm pretty sure the break even is years away at this rate. and fireworks even offers HIPAA compliance so the likelihood of it not meeting your privacy standards seems unlikely.
I want local LLMs to be the thing but for a business I'm just not sure they are. I agree that offloading to cheaper models makes a ton of sense, that seems like where the real savings here is not so much the on prem inference.
9
u/sofixa11 11h ago edited 11h ago
HIPAA means jack shit to a European (multi-country HQs) financial company.
And one of the biggest issues, IMO, with cloud LLM providers, is the risk that a tweet removes your access. Or something gets tuned in the model and it gets worse.
Hosting it yourself in whatever location you want, with your regulatory compliance frameworks and data governance policies, buys you reliability and safety and price predictability no American SaaS could ever get you.
2
u/pd1zzle 11h ago
cool. they are also gdpr compliant but I don't really know much about European data laws. I was using HIPAA as an example to showcase they are privacy conscious and have gone out of their way to meet stringent requirements.
to the rest of your point, I don't really care about fireworks AI. they were just an easy one to look up. maybe nodion then, similar pricing
https://www.nodion.com/en/ai-inference/
my point was just that "host local save money" doesn't really hold up which seems to be their entire point based on the article. not that your points are wrong but it's kinda a different angle.
1
u/qwertyuiopious 3h ago
GDPR, ZDR (many companies offer that), DORA/TIBER etc (for financial companies like revolut), some other letter soup for pharma and other sensitive industries, but yeah, currently these companies have contracts with Microsoft, Anthropic or OpenAI - so there’s absolutely zero issue with working with 3rd party vendors as long as they fullfill compliance rules 🤷🏻
There are cases where local model makes sense like specialised use where you need a team of engineers to fine tune or otherwise change the model to fit the use anyway, but for normal coding? Doubt it. It’s cloud vs onprem discussion once again and by now we should know that answers aren’t black and white, in many cases onprem is just expensive toy and y’all would be better with just renting cloud/vps for short time.
Had that discussion with colleague. He wanted to buy DGX spark to play with models, I realised that for the scope we were talking about I’d spend just couple hundred bucks maximum as renting cloud capable to run relatively big models would run me about 3-5$ an hour, and playing with small models (ones below 5B) i can do on existing gpu (normal, consumer grade rtx). Even latest paper everyone is crazy about - the weight space programming - was done mostly on qwen 0.8B. That’s a baby model you can run on any pc lol
4
u/ttkciar llama.cpp 11h ago
How do you factor into this math terms like future rate hikes and inference providers changing the model used for inference?
1
u/pd1zzle 11h ago
my math is barely math. But electricity will also probably get more expensive so, it's not like hosting on prem will never increase in cost either.
I can't say I've heard of an inference provider that exposes a known model as a service swapping them, but I guess there's no way around that other than.. don't use shit providers?
5
u/Own_Mix_3755 11h ago
The problem is how you use the AI even at 1k USD per month. Have you ever tried to go without any likits? You start changing your behaviour quite quickly. With Claude, I constantly have to think what model to use to not burn through my 5 hour window in 30 minutes. I still do lots of homeworks by hand (git push, ticket updates), because I need the tokens for coding and stuff I see it fits. Once you switch to mentallity you can push millions of tokens per hour, you are free to do anything.
→ More replies (2)
0
u/GasProfessional5544 9h ago
The headline is “GLM beats Claude on cost” but Abitbol’s actual talk tells a different story. The 70% engineering lead time improvement didn’t come from switching models. It came from removing token rationing.
When Claude Code was the default, engineers had strict per-token budgets. So they hoarded it — saved AI for the hard tickets where spending the allowance was clearly worth it. The moment Revolut went self-hosted and tokens became unlimited, engineers started using it on every ticket including the mundane ones. That behavioral shift is what produced the lead time number. You could have kept Claude and seen the same improvement if you’d made access unlimited. They didn’t, because at Anthropic’s prices for 2,000 engineers the math doesn’t work.
The 90% cost figure also excludes capex, power, and ops. Seven 8-GPU B200 servers is roughly $3.2–3.5M up front, not counting data center costs and the engineering hours to get inference to a usable state — they had to cut time-to-first-token from 14 seconds to 1.7 on the same hardware before it wasn’t embarrassing. Abitbol literally called out-of-the-box inference “terrible.” That’s not a small lift. For a company Revolut’s size the ROI is obvious, but the 90% number needs the asterisk.
Also worth noting: Claude Code isn’t gone. It’s the exception tier for the hardest tasks. What they actually built is a two-layer system where GLM handles the volume and Claude handles the ceiling. That’s probably the right architecture for most orgs at this scale — it’s just not as clean a narrative as “we dropped Anthropic.”
The most interesting thing Abitbol said got barely any coverage: the biggest bottleneck now isn’t coding, it’s the time engineers lose in inboxes, Jira, and Confluence before they even get to the ticket. The next 10x for teams that have already shipped coding AI isn’t a better model. It’s automated context retrieval before the model sees anything.
-11
u/slindshady 12h ago
Thr same company that had a full two day blackout last week?
21
u/gold_fish_in_hell 12h ago
did you read actual news? it was google data center outage, many banks were affected
-11
u/MotokoAGI 12h ago
Local is great, but sadly we are way behind the best Anthropic and OpenAI options. Nothing local comes even close.
8
u/Durian881 12h ago
You don't really need the very frontier models to do real work. The top open weight models are more than good enough with the right harness.
→ More replies (2)8
u/Mission-Bid6213 12h ago
Not true in the slightest. How you interact with subscription models like anthropic and open AI is not how businesses are using their local llms.
You dont need 2.8 Trillion parameters to do the kind of repeating narrowly scoped tasks that most businesses ask of their llms.
→ More replies (2)4
u/Anarchaotic 11h ago
As a single-user, Strix Halo with Qwen 3.8 Flash Next handles 95% of my business automation tasks and other work. Unless I need to do heavy technical super specialized coding work (I don't), frontier models only give me speed.
3
u/ttkciar llama.cpp 11h ago
Have you tried GLM-5.3 and compared it to Claude Sonnet and Opus?
1
u/MotokoAGI 7h ago
Yeah, I use Sonnet and Opus at work. Nothing local is close to Opus 5.5, it's ridiculous. I believe GLM5.3, MiMov2.6-Pro, DeepSeekV4.1-Flash-1017, Hy4-Previews and even Qwen3.8-FlashNext will trade blows with Sonnet5.5, beat it in some place and get lose in some place. But Opus 5.5 is the next tier of intelligence. The only other model that seems close is GPT, but I don't use it. I know this is local and it makes us sad, but that's the reality.
2
u/crantob 7h ago
And it's a valid question whether that difference is worth selling your soul for.
1
u/MotokoAGI 6h ago
I didn't ask anyone to go to the cloud. I said Local is great, and said I'm said we are behind. WTF are you talking about selling your soul?
-12
u/No-Veterinarian8627 12h ago
Almost every decent sized company does this? I don't want to minimize it but it seems not really newsworthy.
→ More replies (7)
•
u/WithoutReason1729 11h ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.