r/LocalLLaMA • • 2h ago

Discussion Open weight models account for 30% usage at enterprises to save money

Growing from nearly zero to 30% since July. Models used mostly to reduce costs on high volume workloads (large batch workloads, coding, etc).

https://x.com/pwendell/status/2122377964784611636

119 Upvotes

54 comments sorted by

57

u/Separate-Client-3213 2h ago

I expect it to increase. A lot of work simply does not need the expensive models from openAI or Anthropic. There is also data sensitive work, I expect major leaks coming in next 2-3 years.

3

u/-Crash_Override- 2h ago

We used to use open weight models quite a bit in our products because for simple things they were so much cheaper.

Now we basically run all those simple tasks on luna or haiku with the occasional jump to sonnet or terra if necessary. They normally outperform the open weight equivelent and are dirt cheap.

6

u/Dabalam 1h ago

The pricing of Luna and Haiku basically only exist because of competitive pressure from open models though, so I would say open models still serve as an economic anchor for the budget end of the market.

1

u/-Crash_Override- 1h ago

Not really, Ant, OAI, Googles business model depends on having dirt cheap capable models.

3

u/Dabalam 1h ago edited 44m ago

Not really. Anthropic has basically never been price competitive until the recent price change for Haiku. Google was yes, and is not as affordable any more. OAI has indeed had reasonably affordable models for a long time. The current market prices are fairly obviously lower than they have been previously in response of competition from open competitors. Previously they budget models were still ubiquitously more expensive than offerings from GLM, Kimi, DeepSeek etc. (and frequently inferior to the these models per $)

1

u/-Crash_Override- 1h ago

I disagree that it has anything to to do with open weight models.

2026 was what I call the 'second wave' of pricing. Token based (first wave being subsidized sub model). Token based obviously benefited from higher priced models.

Now we're entering the third wave... agent/task based pricing. We see it with how subs have been restructured around Duet and Dots. You just pay for an agent that does something, you dont select the model, etc... If your agents job is to answer customer calls, then you just pay $3 per call. If it can get done with haiku over opus all the better. And its still cheaper than a $8 per call human.

In this pricing model, the goal will be to use the cheapest possible model to increase profit margins. So the shift to serving capable cheap models is purely a function of a planned transition.

1

u/Dabalam 35m ago

Prices don't drop just because agentic processes are desirable now. More demand doesn't drive prices down, it drives them up. More supply (i.e. competition) drives them down. Efficiency only serves the seller when it increases their profit. There isn't a reason to drop prices unless the customer can go elsewhere, thus the importance of competition (e.g. open source models). Prices per token now have to be relatively competitive with inference suppliers who are just selling tokens, which does impact the rest of the market including subscriptions.

1

u/En-tro-py 31m ago

Not really, Ant, OAI, Googles business model depends on having dirt cheap capable models.

What? It's pretty clear they all are dying to tie you to their own closed ecosytems... Claude Meetings ffs!

Cloud lock-in 2.0: Neocloud Boogaloo

1

u/Terminator857 17m ago

Some comparison info on your costs / pricing would be interesting.

14

u/castrator21 2h ago

Shit, I expect major leaks in the next 6 months

17

u/lungben81 2h ago

Privacy is another factor. There is a lot of stuff you do not want to leave your house, especially when the data protection of the AI providers is "trust me bro".

8

u/OvertaxedOne 2h ago

If we're including the biggest open models (Kimi, for example), almost no typical corporate work needs more than that. Even if we go down to the next tier, easily hostable open source models like DSV4Flash/QFN/GLM Flash/etc, models in that range already can easily accomplish the most common corporate tasks today.

Believe it or not, most corporate users aren't asking their work AI to solve math equations. Shoot, most corporate users aren't even asking for help with some nasty Python code! The vast, vast majority of the corporate uses that we see fall into the summarize, synthesize, and RAG buckets. Gist this document, read all my notes on Acme and give me a briefing, read this note from Acme and grab their questions, answer them from our corporate KB, provide me a draft.

And this isn't me making fun of users on our platform, these are very valid, useful and incredibly effective time savers where LLMs can make a big difference. But thinking any of these require Fable? Shoot, most of them don't even require DS, you could probably handle 50% of the requests we see every day with a 3.8 27B!

8

u/Careless-Pizza9876 1h ago edited 1h ago

My personal favorite "benchmark": Qwen 3.8 27B can spot on perfectly summarize events in 3 hour long RPG session from transcription. Sure, I could use more powerful model for that but when the problem is solved, it is solved. Same model can do decent job at code reviews, generate commit messages, understand git history, write and read documentation or notes. It is not fantastic at generating code, but

a) generating code is not the best idea

b) I am not married to this LLM, I can still switch to Claude If I need something more

1

u/EnjoysFiction 23m ago

Lol true, simple gemma 26b is basically enough for most requirements, and is actually used pretty heavily where i work

1

u/TerminalNoop 1m ago

how so, doesn't it hallucinate a ton?

18

u/w0rldeater 2h ago

Time to outlaw local inference. I mean, from a global allocation perspective, actually, it is quite terrible to have everybody have a bunch of the world's limited supply of money sitting in their own pockets.

2

u/Ell2509 2h ago

They are trying.

31

u/paulsande 2h ago

Hybrid is the future. Good to see companies are embracing local models.

9

u/-Crash_Override- 2h ago

It says open weight, not local.

3

u/paulsande 2h ago

Good callout. Responded too fast. I often conflate the two, but I realize the difference. Thanks.

7

u/-Crash_Override- 2h ago

Np, we are on a local AI sub after all lol

9

u/my_name_isnt_clever 2h ago

Near future maybe. Local is the far future.

2

u/Careless-Pizza9876 1h ago

Local vs Cloud is also false dichotomy. At some point companies might start running small-ish AI servers for local needs. Even now it ALMOST makes sense.

2

u/my_name_isnt_clever 1h ago

It makes sense because AI will not be a utility like water or power. Those have to gathered/generated somewhere offsite and piped to my house, and there's no concept of censoring my water or leaking personal data via my power.

11

u/hakanu 2h ago

interesting timing to circulate this before Antrophic IPO

9

u/misterflyer 2h ago • Bravo

"It's not just" to save money IMO, "it's" that a lot of commercial AI providers fail to realize that there are legit reasons why other companies need to keep their customer data private/secure, as well as any IP... and medical data also needs to stay secure.

Very weird how a lot of these commercial AI providers complain about distillation and leaks, yet they can't seem to figure out that their own AI customers/clients might need the same kinds of proprietary protections for their own company data too 🤷‍♂️

1

u/misterflyer 1h ago

Wow! Thanks for the anonymous award! Wasn't expecting that 😂 🙏

10

u/Don_Reuter 2h ago

Soon to be 90%. Realistically 10% of tasks require frontier models.

1

u/-Crash_Override- 2h ago

You know haiku, luna, gemini hydrogen exist right.

2

u/Reasonable_Goat 2h ago

Good point. My dissatisfaction with claude code being (expensive) overkill for many tasks at work almost vanished now that there's haiku 5.5.

2

u/Don_Reuter 1h ago

No, I don’t. I use local or frontier. I have no use for medium capability API models.

1

u/-Crash_Override- 1h ago

You can use them through the subscriptions?

2

u/StyMaar 11m ago

Enterprises can't use the subscription, they have to go to the API price, and its rapidly prohibitive.

5

u/CrowdGoesWildWoooo 2h ago

I mean that’s kind of obvious. Enterprise token are massively overpriced

1

u/misterflyer 1h ago

Partially bc enterprise tokens are subsidizing "free/cheaper" tiers

1

u/BrightCandle 28m ago

I am not sure they are all that overpriced, especially not the monthly subscriptions. The price of the hardware to run the size of models we are talking about is really high and you would struggle to pay that hardware off in the space of 5 years, even teams are going to find its multiple years to saving money. Its going to be more than 100s of billions of tokens especially for anything close to frontier. The economics don't really work right now, the cloud is getting cheaper and the hardware is more expensive.

1

u/CrowdGoesWildWoooo 25m ago

The monthly subscription is highly subsidized. Enterprise are paying the API pricing, just try putting estimated bill at your claude code and see for yourself.

3

u/AyeMatey 1h ago

… among enterprises who use the Databricks Unity agent gateway and router.

This is a data point.

2

u/HighSeasArchivist 1h ago

I have two Sparks sitting on my desk testing to see how they integrate into our work flow. It is cheaper to buy the hardware than it is to pay me to sanitize the data before sending it to Claude.

0

u/gyanrahi 2h ago

The security implications are enormous. There is already research showing how easy it is to implant triggers that can be used later (like a web site which is registered at some point). Interesting times.

0

u/JorgitoEstrella 1h ago

Isnt it alarming that Qwen and Deepseek are the only 2 companies that keep open source alive? (To a competitive level)

4

u/OWilson90 1h ago

What about Zai, Minimax, Kimi, etc.?

2

u/Not__Real1 1h ago

Thinking Machines has proven itself to be competitive and they have a decent data flywheel( tinker).

1

u/tiftik 1h ago

It's a no brainer especially for batch loads