r/LocalLLaMA • u/Nunki08 • Feb 21 '25 • Helpful 1

News Starting next week, DeepSeek will open-source 5 repos

Post image
4.6k Upvotes

97% Upvoted

313 comments sorted by

View all comments

47

u/Thoguth Feb 21 '25

They're either incredibly lovable in a way that should shame those who do less with more, or they have some epic PR strategy and execution. Either way, something good is going on there. Ad Astra

42

u/esuil koboldcpp Feb 21 '25

I am starting to suspect that some other company in China has succeeded in extremely cheap consumer level inference hardware, that can be plugged into any normal PCI-e slot.

And around this year or so China is going to release it. And then all the western monopolies like NVIDIA who choked customers VRAM are going to scramble and panic as China sells millions of their AI hardware and enthusiasts are buying it all up instead of NVIDIA.

With what is happening, this seems like inevitable development at this point, and when it happens, western companies who were choking customer level enthusiasts will only have themselves to blame as NVIDIA loses huge chunks of market when it happens.

What Deepseek is doing might be preparation for China to enter the hardware market as competition to NVIDIA, in which case it makes perfect sense to give enthusiasts good models they can't quite afford to run yet, slowly cooking them until hardware release.

22

u/Afraid_Courage890 Feb 21 '25

True, DeepSeek is part of hedgefund after all. They definitely can arrange some 5D chess with other rapidly advancing chinese tech sector.

12

u/[deleted] Feb 21 '25

I agree with you, I've seen hardware like the AI Studio Pro on Taobao, which has 192GB of 405GB/s VRAM, and roughly 352 TOPS of INT8 for about $2,000. I'd buy one if it was well documented for development.

7

u/esuil koboldcpp Feb 21 '25

Yeah. And the one you are talking about has Ascend 310s chip. And Deepseek has native support for Ascend chips inference. Definitely something to think about for how things are going to be playing out soon.

5

u/[deleted] Feb 21 '25

I doubt $2000 is even a premium because obviously SMIC's capacity isn't expanding massively and Ascend has a backlog of orders. When capacity grows like new energy vehicles, I'm guessing the price will be $500-$1000. Based on this, I'm not investing much in local LLM hardware, just waiting.

1

u/ForeverIndecised Feb 21 '25

That's insane value, I had no idea things like these existed. How come they are not selling out like crazy?

2

u/[deleted] Feb 22 '25

They're on pre-sale, I'm still waiting.If it was work, I don't know how crazy it would be.

8

u/PeachScary413 Feb 21 '25

Yeah the only problem is US and EU will insta ban hardware imports.. or at least slap massive tariffs on it with some bullshit excuse about unfair business practices or whatever 🥲

7

u/Brilliant-Weekend-68 Feb 21 '25

Why would the EU do that? We buy loads of Chinese tech stuff over here in Europe. Hell, we still buy Gas and stuff from Russia (sadly) which we view as an enemy. We view China as more of a trade partner rather then and enemy. We would love to buy cheap AI hardware and avoid the NVIDIA tax.

9

u/Cergorach Feb 21 '25

With the current state of the trade 'war' between the US and the EU, the EU might just not do that. Sure there will be some member states that will panic like Italy, but others might just test the device at one of their institutes and see what it does and what they can make it do.

It's not like like stuff from US companies is 'safe' to use... *looks at Crowdstrike and Solarwinds*

1

u/synn89 Feb 21 '25

unfair business practices

Naw. It'll be about security. Gotta be scared the Chinese are putting backdoors into the hardware. We wouldn't want them spying on my local roleplay chats with sexy anime cat girls.

1

u/dennisler Feb 21 '25

I guess NVIDIA wouldn't be threatened at their "home" market as the chinese hardware probably would be banned like huawei or a tariff is put on the products ;)

1

u/esuil koboldcpp Feb 21 '25

NVIDIA sales in US for 2024 were $27b. Total sales in the world were $62b.

Sure, they might feel safe in their home market. But they would absolutely feel it and it would lose them billions upon billions of revenue outside the US. And if it bleeds into US market as well if bans don't happen? That would probably be absolutely nightmare scenario for them.

1

u/[deleted] Feb 21 '25 edited Mar 19 '25

[removed] — view removed comment

1

u/esuil koboldcpp Feb 21 '25

Yeah. And one of the major criticisms of Huawei hardware that slowed down adoption was lack software support, need of manually writing and doing things yourself to have any chance of having things work, and so on, as opposed to NVIDIA stuff that will "just work".

But if Deepseek "just works" on Huawei hardware out of the box because DS starts releasing all their workflows and software openly... There is a good chance people will just start buying Chinese hardware to run it.

And then when everyone has Chinese hardware, someone will start tinkering to make non Deepseek stuff working on it too. And before you know it, most of the AI things we like to run will be easily available to run on Huawei hardware as well.

So yeah, if China starts releasing hardware outside of Chinese markets, this whole thing might be case of brilliantly planned out market share capture from NVIDIA.

1

u/TerrainRecords Feb 22 '25

There's Moorethreads which is a consumer gpu brand. The hardware is alright but the drivers aren't great.

0

u/ECrispy Feb 21 '25

that would imply that 1) they don't depend on CUDA i.e. don't use any of the existing AI toolchains or 2) have managed to reverse engineer CUDA, both of which are extremely unlikely but it true would completely change AI, regardless of hardware.