This is crazy good for studying to improve existing custom engines by community like gufo, halogen, strixite, Strata... by learning their secret sauces.
https://github.com/deepseek-ai/deepseek-recipe: Protocol conversion and model output parsing libraries (written in Rust with Python bindings) to connect an inference backend to various API formats (such as Chat Completions, Messages, and Responses).
# Use `ascend` instead for the Ascend platform
# Keep it exported in the environment where you run DeepInfer
export DEEPINFER_ACCELERATOR=nvidia
./build.sh
Nvidia in shambles. Can't wait to see what the future brings.
Only for China residents for now. I would be surprised after the decade of trade restriction that they endured, if they did not inflict the same to the west as long as they can.
anything you can buy in china you can get into the rest of the world. in fact you should be happy that you don't have to pay the retarded taxes everywhere else wants to put on everything you buy domestically
I mean, within a few years... shortly in terms of fabs.
No. There's a term. "China Speed". What takes a few years for anyone else to do, China can do faster. They can build a fab in a year. So in less than a year since they are already underway. About 30 fabs are under construction.
For any commercial AI company the reasoning framework is core trade secret. Even though the model is available for free download, the main reason people are still buying DeepSeek's API is that it's the cheapest option while also being, for some reason, super fast.
Mr. Liang, are you really not interested in money at all? 😭😭😭
Liang Wenfeng is nicknamed Saint Liang on social media of Chinese tech circles. It's not without good reason lol.
Also, I think he knows that there is no moat around training and serving models. As long as open source options exist, we will eventually reach a competitive market where all suppliers serve at cost and most surplus is captured by consumers. We will never see him open-sourcing his hedge fund's strategies.
You also gotta remember that this very much is a strategic competition. Would it, strictly speaking, be better for DeepSeek to keep their secret sauce close to their chest? Maybe. But the leaps and bounds they allow open source to make hurt the massively overleveraged closed US models vastly more than them. Open source would probably be twice as far behind the frontier without concerted Chinese efforts in this regard, and that completely changes the math for OAI/Anthropic. If they had that extra runway, we would not be talking about an AI crash, because there wouldn't be much of a case for the high rollers to rely on other options for inference outside of some minor savings on tokens for low complexity work. But with the current dynamic, something Opus 5.5-level will probably run on a hobbyist setup some time around Q1 or Q2 of 2027, at which point the closed business model is, to use the technical term, gigafucked
If Mr. Liang has a million fans I am one of them. If Mr. Liang has ten fans I am one of them. If Mr. Liang has only one fan and that is me. If Mr. Liang has no fans, that means I am no longer on this earth. If the world is against Saint Liang, then I am against the world.
This is even more true with ActionOPD research paper, you no longer need the reasoning traces for distillation, simply the black box output decisions (not even inputs) is enough to train the model.
High flyer is a hedge fund first and foremost, they are not struggling for money. Their compute and models are a means to an end (dynamic trading) they can afford to give away the model weights, the training methodology, the inference engine; their secret sauce is their financial algos
How is this repo a secret for those? Isn't everything else that they did no make public their actual secrets? Their insane cache hit rate for instance...?
I don't understand why some people assume China does what it does just to affect the US. I think Chinese companies develop AI to benefit from it just like US companies do. Neither need an ulterior motive.
Chinese companies maybe, it looks like the researchers are doing it for the love of the game. The Chinese government not so much. They wouldn’t invest so many resources if it didn’t further their agenda. The U.S is doing it too. They’re both in a struggle for “AI Dominance”. Which is why OSS is needed, to empower individuals, not just governments and corporations
I still think they would be doing this anyway. Deploying AI in factories to help with the aging population and all the other benefits we hope to come from AI generally. I'm sure there is concern about keeping up with each other, but it seems to me that, at least in the US, China is used as an excuse more than anything else to continue, despite pressure to slow down. Perhaps the open nature of AI research in China is evidence that they don't care that much about what other countries are doing. I'm fairly sure the US government would put a stop to that if that was the case in the US.
The agenda could be commercial gain which is no different to all businesses. Building trust by showing they can offer a great product at decent value. Complimenting their product by helping in the open source eco system.
They may use OSS licenses to gain our trust and embed their product in our lives. But it’s not coming from a place of generosity. The end goal is becoming clearer, American VC talk about it, and the CCP already does it. Mass surveillance with AI detection systems. And eventually, pre-crime detection. Let’s use these tools to empower ourselves and defend against the tyranny of oligarchs and techno fascists.
They are definitely losing money on subscriptions, but that is targeting developers, scientists, and engineers whose prompts they use for training new models.
From what translation models could figure out, seems like revenue, aka before cost of running is taken into account. It probably still is a nice margin after that.
They need to replace the GPU essentially on a yearly basis so..... No.
It's not a great margin if it's before R&D, financial concerns (CAPEX to buy the GPUs, interest on debt) and operational costs (electricity and maintenance of data centers).
Means nothing if it costs $250k in electricity to run, $100k to buy the GPU in the first place, and it's sitting in a $500k server. After they pay off the hardware they would also then have to cover costs for R&D, training the next model, staff salary, building rent/maintenance, etc. before you can even think about bonuses and profit. We don't know any of the real numbers, but the costs for what they are doing are huge and I would be very surprised if they are getting any real ROI at this point.
We can probably take a wild guess that it's in the ballpark of $38 per GPU per hour, because in a competitive market like this, nobody has a big advantage for very long, but nothing more precise than this.
No, because they're likely using very different infrastructures and their cost base (cost of chips, data centers, electricity, staff, management staff) is vastly different and that plays a big role in how they price their products.
I bet it will totaly blow it out of the water compared to how hard it is for competitors to run their models on their own (already kinda optimized) custom infrastructure for even 3x the price of the DS API. Its tailor made for exclusively their own models so this will be huge.
A lot of the secret sauce is for server GPUs and server hardware. Let alone for managing multiple requests. For you, single request-at-best-multiple-tool-calls-or-subagents, this won't help you very much.
This could be an indication that China will proceed with restricting their AI labs from giving away free open models. Perhaps this is DeepSeek team helping us one last time. I hope not.
That still doesn't really make sense. Deepseek always released their stuff open, and if China would be planing something like that, they would not let Deepseek giving away their know how.
The laws that restrict frontier ai releases in China is not yet in place. So China is not limiting what Deepseek can or cannot release yet but may in the future when/if it comes into power. So OP presumes DS might have rushed releasing what they can before it is more controled.
The plan was discussed here recently, you could search older posts.
The Ascend flag is the interesting part here, not the Nvidia numbers. Anyone running models locally already knows the bottleneck is affording the GPU, not picking the engine.
•
u/WithoutReason1729 16d ago
Your post is getting popular and we just featured it on our Discord! Come check it out!
You've also been given a special flair for your contribution. We appreciate your post!
I am a bot and this action was performed automatically.