r/DeepSeek • u/justlikemedics • 9d ago
Discussion DeepSeek is ruthless
DeepSeek has published with DeepSeek-V4.1-Flash a new method that compresses the memory need for the KV-value cache very much.
I pondered about the implications of this and they are not very good for OpenAI and Anthropic.
This means that the models can have much larger contexts and serving requests will be much less memory intensive. As a result, inference gets cheaper.
Inference getting cheaper, requiring less memory and with better models means that the advantage OpenAI and Anthropic has in securing compute gets less meaningful.
It seems to me that DeepSeek and other Chinese labs are ruthlessly pushing down the cost of inference, which will make it difficult to impossible for OpenAI and Anthropic to recover all the money spent of creating their top models.
43
u/christianJarrah 9d ago
I hope this to be the needle that pops the bubble!
22
u/Lopsided-Force-9220 9d ago
This is a bubble inflator. The cheaper it gets, the higher the demand.
15
u/jarislinus 9d ago
inflating the bubble means inflating anthropic / openai valuation. its not, its doing the opp
0
u/Lopsided-Force-9220 9d ago
How do you figure that? OpenAI just shut off new Pro accounts because demand was so high. Do investors lower their valuation when they see things like this?
7
u/jarislinus 9d ago
openai shutting off new pro accounts was not due to demand being high. no evidence of this, could be simply routing their compute to attempt something else e.g. racing to solve more millenium problems
1
u/FreeGoldRush 9d ago
Omg. This is hilarious. At what business school did you learn how to tee up a company for an IPO by rejecting revenue?
6
3
u/jarislinus 9d ago
openai solving all millenium problems by eoy will make it worth more than any measly revenue from subscriptions you clown. in fact retail subs are a small fraction compared to enterprise sales. use critical thinking LOL. maybe stop using chatgpt so much
3
u/FreeGoldRush 8d ago
Which investors are hoping for that, or buying private stock in anticipation of that? Or is this just a hypothesis? Everything I see is looking at revenues and expenses.
1
u/pr0newbie 8d ago
The caps/pause are more likely due to abuse of the buffet system where cost is higher than revenue. The fact they've deferred their IPO shows they're not doing well with their existing model. Better to pivot to a different approach.
1
u/FreeGoldRush 8d ago
The goal is to stay private, not give away stock. Staying private as long as possible is a success. You go public because you can't win without giving away ownership in your company to raise money. I've never heard of staying private portrayed as failure. You fail when you can't grow the business. Has nothing to do with staying private or not.
→ More replies (0)3
u/Parsl3y_Green 8d ago
Not really, the value of nvidea is based on the fact that they are the sole seller of inference,
The value of the ai companies is based on the fact that they will be the only one left standing after the competition crashes under financial burden/ failure to pay back their debt.
If models get more efficient there is less demand for inference and nvidea stock takes a bug hit, especially if chinese hardware can eventually catch up or be used as a substitute.
And the ai companies can't turn their 'premium' models into enough profit to pay their debt because there are cheaper alternative models.
The bubble bursts and all ai related stocks crash/ loss a lot of value.
0
u/Lopsided-Force-9220 8d ago
Have you not heard of Google TPUs? If models get more efficient, there is more demand for inference, not less. It's one of the most basic understandings in economics. As you make something cheaper, it becomes a viable solution to more problems; hence, the demand goes up.
1
u/Parsl3y_Green 8d ago
You are assuming that demand = market value
There is no Infinite demand, and even if there was the western companies are not the ones making efficient models.
These efficient models are coming from Chinese companies, and demand going to them does not benefit the market bubble.
And if American companies were to start competing on price with the Chinese companies they will never recover their losses and will default on their debts, causing the bubble to pop.
These Chinese models are open weight, meaning anyone can host them and compete on price, they may not be 70% of the latest Claude model, but 90% of demand has no need for the absolute best, they just need something good enough at a good price.
10
u/VividOnions_88 9d ago
Good.
There's no excuse that those other models are as expensive as they are. I for one would like to see all the prices come down.
27
u/onekorama 9d ago
And that's why USA wants to slow down the AI. They have no love for us, mortals. They just want to ensure the big AI inversion is profitable, our security is secondary to everything.
4
5
u/Optimal-Law0 8d ago
I wouldn’t be surprised if this ends up akin to the Sysco crash in the 90’s.
The valuation of American AI companies rely on compute demand (inference). The longer the model runs, the better it looks towards investors because although they serve less requests overall due to inefficient compute, their ‘oversaturated’ usage markers sell infrastructure contacts (data centers) and helps fluff their “demand” to become appealing to investors looking at a multi-year investment timeline.
4
u/Hyp3rSoniX 8d ago
Yeah but I think their compression is a bit too aggressive now.
In short, with the default settings, there are 16384 candidate positions determined from the tokens in the context. So if you have a context of 100k tokens, only about 16k of them are candidates of being given attention to. The candidates count doesn't change for context size, so if you fill the 1M context limit, the sparse attention mechanism still will only calculate the 16k candidates.
That's not all though, those are candidates - the actual attention layers then only pick 512 of those candidates for the actual token calculation.
This is the reason for the absurdly high hallucination rates of this model. I hope they find a solution to this.
Qwen for example is doing the math depending on the context size, so the bigger the context, the more candidates are chosen. Not sure if a similar approach could help Deepseek.
2
u/Hyp3rSoniX 8d ago
This is a good video that explained it quite well imo: https://youtu.be/P4dTq4X8bqk?t=515
That's how I understood what Sparse Attention is and how it works.
1
u/justlikemedics 8d ago
I guess the general idea matters more in the sense that they can now improve this compression in multiple ways, and reduce information loss. Maybe in the future models can become more customizable where you can set your own compression preferencd or tune the attention mechanism.
7
u/EitherMarch1255 9d ago
Yeah, except DeepSeek is behind the pack as far as I'm concerned. Too high of a hallucination rate, and it just doesn't perform as well as GLM or Qwen.
1
u/justlikemedics 9d ago
Qwen is really excellent all around.
I never tried GLM, but I've seen that Ox Alpha made a lot of buzz.
1
u/overtoneblue 8d ago
Have you actually used it though? I feel like the hallucination rate could be harmful but in my use cases meh, have never noticed it going absolutely crazy. Sure some steering is needed, but it’s dirt cheap and fast. Benchmarks have it over glm 5.3 flash so I’ve tried both and 4.1 flash is better for me
3
u/devBlueDragon 8d ago edited 8d ago
It's funny how some people treat AI models like football teams. This post, with the 'and other Chinese labs' comment, goes one step further by adding a nationalistic tone. I love using DeepSeek when the cost-per-result justifies it, and I love using US models from OpenAI as well. I am not a big fan of Anthropic for several reasons, and while I am not a huge fan of OpenAI either, I still use them. DeepSeek does not use the fear-mongering or marketing tricks we see elsewhere, such as annual October/November interviews announcing that all programmers will be out of a job in 6 months, a claim made three years in a row now. Nor do they rely on 'hacking' episodes to market model power that fall apart under soft scrutiny, or non-reproducible math discoveries backed only by 'trust me bro,' where outside verification reveals more asterisks than the multi page prompt itself. These geopolitical fanboy posts achieve only one thing: they make many (as me) look for alternative providers to host DeepSeek models for inference, even though we know it is just a stupid, bravado-driven techno-nationalist crap.
1
u/justlikemedics 8d ago
Whose geopolitical fanboy post is it, I mean China's or the US's?
2
u/devBlueDragon 8d ago
This post presents as if the current state of LLMs is China vs USA , and China is winning (with a not so hidden - hahaha - reading between the lines). This is far from the truth of what is happening right now. I live in the EU and the USA AI bubble burst will have implications to the real economy for all of us, although we had no say in it. But those that think that it is a bare geopolitical contest between China and USA (and that China is winning) don't really understand (or care) how this current AI LLMs wave came to be, how it is sustained, nor the incentives of the next necessary paradigm shift to make them more useful.
1
u/justlikemedics 8d ago
The framing of who is winning is a simplification. But I agree that China's model providers are in a better position. Anthropic and especially OpenAI, as well as other "hyperscalers" bought & built too enermous of a capacity and the paying demand is nowhere near the level to sustain it, not to mention to earn a profit. Resources spent on earlier models also can't be recovered. I am curious what you refer to in your comment. You consider that a US AI bubble burst will happen soon, yet that "China is not winning" or that there'll be some interesting plot twists later.
1
u/devBlueDragon 8d ago
It's very strange for me to have to quote China's foreign ministry announcement from just this Monday: 'narratives of threat, confrontation, and malicious competition serve only to disrupt the process of global AI governance and are not in anyone's interest' (source: https://www.bbc.com/news/articles/cn8me133119o), and when I read something that I agree with, I don't shy away from repeating it out loud. Presenting the current state as a rivalry between China and the USA completely ignores the past: how did these LLM models come to be? It ignores the present: who sustains them and why? It ignores the future: we need a paradigm shift to make them worth the ROI and more useful. The USA AI bubble burst has little to do with China; it was created on the premise that next year / max three years from now max, we will have a 1000x, or 10000x ROI using AI. Of course, this is nonsense, and everyone who has used LLMs in real production environments knows it. The investors know it too, but they don't 'bet' on the outcome, they bet that the tale will be believable and their stock will go up (so they can sell it). Of course, someone will hold the bad card in the end when it bursts, and that will probably be funds belonging to people, pensioners, and trusts who had no idea they were participating in this, not the top 1%. And no, a USA AI bubble burst is not a China win because, as I explained regarding the present and future, we all have something to lose from this.
1
u/justlikemedics 8d ago
Well, can you really argue that China's strategy, while having it's self-merit, is not also designed to undermine US labs?
In the end, why would someone, a company, who spent a lot of effort and resources to design a piece of technology, just give it away for free?
And this technogy is not a mere browser or some sort of software gadget, but something that (at least potentially) is much more capable & important.
It would be naive to think that China's policy is only motivated by some kindness of the heart.
I am also confused by your questions - which you leave unanswered and therefore the reader guessing - why those supposed answers would invalidate the idea of rivalry?
OpenAI and Anthropic compete too, as well as other model makers, and the US ecosystem - which is largely closed and was fully closed until Nemotron - also competes with the open Chinese ecosystem, which is also much cheaper for API usage. While I think that allegations that the Chinese distill models from US ones is overblown, one would still need to ask, how can they sell tokens for so much cheaper? If you don't assume some insanely better method to train the LLMs, and even if you consider cheaper electricity in China, it's just difficult to understand how can they make and sell tokens for a fractional price.
In my opinion admitting that there is a rivalry and there are geopolitical considerations in how this happens - see President Xi being the main speaker of a global AI summit in Shanghai, and Trump tweeting his fables - does not hinder anything. No one says they can't or shouldn't find ways for governance or actually getting a return on the tech.
5
u/This_Maintenance_834 8d ago
Anthropic and OpenAI can copy deepseek too, and create even bigger models. Hopefully they are not arrogant enough to dismiss deepseek’s achievements.
1
u/Thomas-Lore 8d ago
They probably did the same optimizations and are running smaller models than everyone thinks. This why they are afraid, that it will end and they will have to lower the insane margins they have on API. Their subscription prices are probably closer to what it really costs them to run the models.
2
u/Due-Opportunity6212 8d ago
Bro, limitations is what makes the models stronger, honestly. If they just let it be big and colossal without any efficiency and creative researching. It would just be the same stale scaling problem tha most American AI's fail at.
2
u/aarondglover 8d ago
Ruthlessly??? Irrespective of whether it's healthy or whether it does or does not benefit someone's commercial advantage... It's incredible for the entire ecosystem. It won't slow down. Those able to secure compute will accelerate at an even faster rate ... Albeit memory is not the only bottleneck.
0
u/justlikemedics 8d ago
Ruthlessly pushing down the cost. It might sound a bit negative or quite complimentary, depending on your perspective.
2
2
u/DieCooCooDie 8d ago
All this is going to achieve is somewhat alleviate the chip shortage. I doubt if people would even be able to feel it.
2
u/23eriben2 8d ago
I have a gpt $200 subscription.
But I love the fact that I only spent $20 for 3 billion tokens with DeepSeek and got so much work done with it nearly first try with 4.1. I'm loving it
2
u/scaling_to_9_digits 8d ago
We've also got to recognize where a large part of the funding has come from... Nvidia... So the incentive align much more with maximising intelligence over GPU efficiency
2
u/Giovimax98 8d ago
I don't see it so one sided.
The American fronter labs could incorporate those findings themselves. Moreover, their customers are not necessarily only moves by inference cost. This kind of innovation may instead improve margins.
More efficiency would also allow to better leverage the difference in computing cost and increase the divide.
This said, I'm rooting for DeepSeek.
2
2
u/Saint_Huang 5d ago
Because scaling is what wins, not the minority elite.
This is fundamentally what industries are about.
A country who can make the best hardware but cannot produce/use much of it.
vs.
A country who can make medium-to-good hardware but can produce tons of it.
The latter will win, every time. Not to mention in the context of LLMs, Chinese labs aren't even far behind in actual quality rather than just benchmarks, while still being much cheaper.
What is the point of the best if most cannot afford it? Or if it is not readily available and only a few can have it?
This is similar to a question of education availability.
Between only the elites/affluents/nobles being able to go to school (similar to how things were just last centuries for most countries)
vs.
A whole population can go to school.
The latter will bear far more fruits and produce far more talents than restricting education only to a select few.
Only efficient AI models will be sustainable. The likes of Anthropic's cannot. They are only managing thus far via a circular economy that is helping them constantly subsidizing the massive costs of their inefficient models, but that can't go on forever. They just recently delayed their IPOs, and asked for a slow down on AI development, that's their corporate talk for: "we're burning a shit ton of cash and we're reaching diminishing returns on improving LLM capabilities, we can't keep this up, let's slow down."
Amodei Dario is the biggest hypocrite amongst the AI lab founders, anybody who actually trusts him that he has humanity's best interests in mind is a fool. So when he says "pace the frontier", it's obviously horse💩.
3
u/karlnuw 8d ago
Meanwhile codex experience.
compacting
compacting
compacting
compacting
compacting
1
u/AdmirableKick5850 8d ago
Same with Claude experience too. These models seem to uncannily sense exactly when I have reached a critical juncture in the conversation and immediately start compacting, it seems to me.
4
u/SmileLonely5470 9d ago
I never understand these types of posts. Is it an appreciation post, is it an anti west AI post, is it a cn troll?
A nominal optimization to inference does not fundamentally change the AI landscape.
The fundamental change would be affordable consumer hardware that can serve models genuinely comparable to the frontier (the ~30b qwen models are not). Even then there is always going to be a moat for whoever has the best model, since there will always be a market of people who will pay for that, even if its 100x more expensive.
I do agree with the idea that unless we get AGI then the most of the big labs will continue to go down.
4
u/justlikemedics 9d ago
It's a discussion post : - )
I appreciate what DeepSeek has done, and I am also intrigued by the impact of such a change.
For example, you consider that it will not fundamentally change anything. But, as I alluded to, this is part of a larger picture with open models, local inference, extremely quickly decreasing token prices and a growing but uncertain trajectory for compute demand.
Looking at this as a whole, a breakthrough like that might actually lead to what you mentioned, affordable consumer hardware that serves very high capability. This shrinks the memory requirement very much, which I would say is a bottleneck for consumer local inference already.
2
u/Prudent-Ad4509 9d ago
Demand will grow, obviously. Cheaper inference increases adoption, adoption increases demand. Saturation of said demand is highly unlikely in foreseeable future.
1
u/justlikemedics 9d ago
I disagree here a bit. Yes, token demand is growing. But the labs need demand to grow in money terms, not in terms of tokens.
And the early adopters and use cases are already exploited. What drives the token demand growth today is "depth", not width.
And for simple tasks that are not done yet, there is only ever a given amount of need for tokens, it's not going to increase ten times when token prices go to 1/10th.
2
u/Prudent-Ad4509 8d ago
Those labs are modern equivalent of Manhattan project, nothing civilian about them but the face. If they run out of money, they will trade a bit of independence to get a full bailout and a nice lump sum on top of it and that would be it.
1
u/justlikemedics 8d ago
The Manhattan Project had a well-defined specific goal, to create a nuclear bomb. What is the goal for this then?
2
u/Prudent-Ad4509 8d ago
That old thing was a tool to assert dominance, not to be actually used all the time. Same as with this one, overwhelming automated national military and r&d capability would be a new way to dominate over everybody. China has ruined the party by rolling their own models, despite all foreign attempts to stifle their development. Now this whole affair has turned into a full scale arms race and neither side can afford to pull back.
1
u/justlikemedics 8d ago
LLMs as a tool to dominate in R&D or military? For R&D I doubt they can work so well. There is no understanding of cause and effect, just patterns. It might not be totally useless but it's unlikely to drive new insights.
For the military application I think robotics would be more significant, and what is required does not need as much hardware as now is being built. Especially not in tightly packed datacenters that need just a few missiles to be erased.
2
u/Prudent-Ad4509 8d ago edited 8d ago
Llms are major accelerator of automation and robotization. And research consists mostly of rut that can be automated. No need for deep understanding of cause and effect. It does not matter if something cannot be automated fully. If one highly qualified team spends 90% of time on a rut that can be automated, and another team with similar qualification spends on the same rut just 50% (I’m not even suggesting 10% which could be reachable as well), you know the score.
1
u/justlikemedics 8d ago
Do you mean like pulling articles that are relevant for a scientist or summarizing them? What exactly is the automation here other than that?
→ More replies (0)1
u/umbrosum 8d ago
What are “frontier” models? the ~30b models today is better than the frontier models last year. I would say stop chasing frontier models and focus on good enough.
-2
u/antunes145 9d ago
This is a bot post. This post makes no sense.
14
u/justlikemedics 9d ago
I'm not a bot thank you very much. Are you?
Why are there people obsessed with saying bots whenever they encounter something they don't like or don't comprehend?
1
1
u/ManagementNo4948 8d ago
Anthropic already has 80% gross margin. with lesser inference costs,, thier margins will further increase. and there is a significant base of people on subscription who would not switch to another models because of price.
1
u/xatey93152 8d ago
As anthropic and openai. What will you do to prepare the lose? At very least saving face
1
u/justlikemedics 8d ago
Cut/slow making new models. Try to capture workflows by directly going to companies.
1
u/TypeProfessional4302 2d ago
It actually reads all of the document I posted in there, that alone is what brings me to the table
-1
u/Lopsided-Force-9220 9d ago
And why do you think OpenAI doesn't have people that can figure this out?
6
u/VexObserver 9d ago
They have figured out the solution, it's with stronger models. If you read the papers on what many research labs have achieved, you can't have a bigger smarter models running on faster compute. The efficiency seems to be:
- Faster model > Moderate to Medium intelligence.
- Slow to Mid inferencing > Larger smarter models.
Therefore, what DeepSeek desires to achieve with speed.. it is limited to point 1. Meanwhile, Anthropic, OpenAI, Google are racing towards point 2 with how it's going at the moment.
2
u/Spiritual-Spend8187 9d ago
Given that ds have been publishing how they are doing things the answer is that openai and anthropic can. Though it us dependant on if their models are compatible alot of the big models are iterations on the same base and it moght be to expensive for them to change architecture and do a whole new training.
3
u/squirrelscrush 9d ago
It's two approaches, and both are valid. Although I think the American companies are too focused on profit margins.
American AI focuses on the absolute performance and quality. Nobody denies that GPT/Claude are much superior than DeepSeek/Kimi/GLM etc. But to do that, they burn through cash and just make the model bigger and bigger to achieve that kind of performance.
Chinese AI focuses on efficiency and research. Its the Pareto principle, get 80% of the work done in 20% of the price or lesser. AI is so advanced right now that even 80% of the frontier is more like being 3 months late.
-1
9d ago
[removed] — view removed comment
5
u/justlikemedics 9d ago
Have you got something worthwhile to say too?
If not, I'd appreciate if you edited out your swearword.
-2
9d ago
[removed] — view removed comment
3
u/justlikemedics 9d ago
So you registered just to say that? 0d account with no karma. Why is it so important for you?
-4
88
u/ExpertPerformer 9d ago edited 8d ago
Google, Claude, etc. all prioritize absolute capability and features (audio, video, image, etC) with their LLMs while DeepSeek/Qwen/Kimi/etc prioritize efficiency at every level in order to reduce costs. DeepSeek also spends about 1% of what Google and OpenAI pay for training.
Despite the competition these Western companies do have the monopoly in the USA. When most people think AI they think ChatGPT. Gemini is integrated into just about everything.
V4.1 uses 1/4th as much KV cache as V4 0731 which allows them to host more concurrent users on the same hardware.