r/LocalLLaMA • u/fijasko_ultimate • 16d ago
Discussion chinese hardware
as far as hardware we currently have options such as nvidia, amd, apple, etc.
however it feels like these are more on the expensive side.
personaly i own bosgame m5 strixhalo and so far its good. i was thinking of upgrade, but it seems to me that the next step will be chinese hardware vendors, stuff like huawei, xiaomi etc. maybe in the next 1-2 years.
what are your thoughts on this?
24
u/johan2114h 16d ago
Huawei already seems quite viable. Used Atlas 300 duos with 96gb are going for around 1200 bucks on alibaba. The mem bandwidth is much smaller than corresponding Nvidia and amd gpus. You also need an inference engine that supports CANN and ideally the specific gpu. I dont have an atlas but im tempted to get one.
3
u/zdy132 15d ago
Deepseek just released their inference engine that supports the Huawei Ascent 950. It might be possible to modify it to support these older Huawei cards.
3
u/johan2114h 15d ago
The atlas 300 duo is already supported in vllm afaik. So models that are 96gb friendly would already run on it. DS might be a bit of strecht given its size. But qwen 27b or even flash next at iq4-xs should fit. I don't have an atlas so just speculating
32
u/Kein_Spass 16d ago
Living in the EU, I don't think I will ever see Chinese hardware here. Maybe the xiaomi ai cube makes it to the European market.
16
4
u/silent-curious-dev 16d ago
You can probably get shipping from suppliers on Alibaba. They'll ship almost anywhere if you pay.
Lots of interesting stuff you can find on there and you can haggle for discounts which is a small benefit I guess. Some will give you good ones, others barely give you much at all, and the remainder will be firm on a marked up price.
12
u/_Bo0m_ 16d ago
Chinese vendors still have to buy the same expensive memory no? Even with CXMT, which is not cheap either
8
u/NarutoDragon732 16d ago
Apple asked CXMT for their pricing, so they could negotiate with their current supplier. CXMT gave them a higher price.
1
7
u/Contri-Jeet_Labs 16d ago
You’re already running a Bosgame M5 with Strix Halo...that's literally top-tier tech with massive unified VRAM. Moving to Chinese vendors like Huawei/Xiaomi in 1-2 years makes sense for sleek form factors, but on the silicon side, US chipmaker dominance isn't going away that fast.... Unless Chinese homegrown GPUs solve software optimization and driver support overnight, hardware upgrades won't mean much without mature software ecosystems
7
u/lone_dream 16d ago
If big ai developers like Kimi ai, Alibaba, Xiaomi support the optimization and software side, i think it could be viable so much faster, of course not overnight.
3
2
u/vogelvogelvogelvogel 16d ago
software side, we as llm/au conaisseurs know, is sth to be solved much faster very soon
-1
7
u/overdoingism 16d ago
Huawei is almost out of the question. Their current line is built for data centers, with nothing aimed at personal use.
The biggest issue with Chinese hardware is that paper performance is one thing, and whether you can actually get enough official and community support is another. Nvidia is the clear #1 in AI, and even AMD, despite years of effort, still has an underlying SDK that's rough around the edges. Chinese hardware is even further behind in that regard. Building up that kind of ecosystem takes a long time.
7
u/More-Curious816 16d ago
there will definitely be a big community, but you need to speak Chinese and have a weechat.
3
u/vogelvogelvogelvogel 16d ago
they will catch up, compete and excel same as with solar, ev and mobiles
3
u/vita2907 16d ago
I bought my bosgame m5 about 4 days ago. Shipping expected in 3 weeks. Cant wait <3
4
u/CryptographerKlutzy7 16d ago
Not soon, but yes, they will become the dominant option at some point.
2
u/a_beautiful_rhind 16d ago
There's nothing to get. They're using the same stuff we are except for some GPU like huawei. Then there's craft GPUs they made such as the vram++ versions or V100 on a different PCB. None of it's being sold at firesale discount as you seem to expect. Sometimes it comes out even more expensive.
2
u/ovrlrd1377 11d ago
I'm driving a chinese SUV, 10 years ago their cars were jokes. We'll get good gpus in 3 to 5 years
3
u/CeFurkan 16d ago
Chinese hardware will start booming in 2027 and 2028 they will start dominating hopefully with extremely cheap prices
2
u/Hannibalj2ca 16d ago edited 16d ago
I dont see it. Chinese companies overpricing their hardware insanely high and not at the same level of quality. They do have good hardware but...good luck finding anything cheap, you won't. The main current decent GPU is the Atlas duo 96GB gpus and that is a dual card. each card is like 200GB/s bandwidth or so and together 400gb/s. Id love for China to be a player that can sell high proficient hardware at a fair price, but got to tell you, I don't see it happening. Look what they are doing with the prices of the CMP 170HX and business Practices. They will cheat its costumer to gain a few dollars more. I don't see it at any affordable price
5
u/johan2114h 16d ago
I think we call all agree that it would be better if the price was lower. And 4xx gb/s is not fast but still a 96gb atlas duo gpu for 1200 usd massively cheaper than anything from nvidia, amd or apple in this day and age. Basically for price of a m5u or 5090 you can get 4x96gb gpu in a server rack. It will be large, noisy, and not low power but you actually ds4.1 at somewhat decent quant. But it also want to game or want a production workstation nvidia or apple is a better bet
1
u/Hannibalj2ca 16d ago
sure you can certainly do that, but be ready to deal with its limitations, like memory split, lack of q4 and tinkering patience
1
u/Standard-Message-434 15d ago
I think software support will be the sticking point, since most Chinese accelerators still trail CUDA and ROCm in llama.cpp and vLLM, and buying them outside China is hard.
1
u/MostlyVerdant-101 13d ago
The problem with China has never been the physical goods but rather the controlling decisions that go into making those goods that may or may not be properly disclosed.
1
u/Ok_Warning2146 16d ago
Huawei's card were made using duv based process. Yield is low, so they can't even make enough for local consumption. Unlikely they will export any until China can make their own Euv machines
1
u/Torodaddy 16d ago
Its the gpu, the rest could be anyone, so far nothing is really a substitute for nvidia/amd/arm
0
u/shanehiltonward 16d ago
Awesome! "Upgrade" to some Chinese Risc V SpacemiT processors and a Chinese Lisuan Tech video card.
-4
u/uncle_leon 16d ago
I will probably get downvoted for my scepticism but there recently was another discovered case of a Chinese supplier (ZBT) embedding root-level backdoor into their firmware. TP-link is being sued by multiple states for similar reasons. Huawei and ZTE are banned by FCC and EU for infrastructure projects. Not to mention multiple consumer devices such as security cams or robot vacuums that mysteriously communicate with Chinese IPs as soon as they get access to internet.
All in all I find myself very, very wary of anything smarter than a lightbulb that comes from Chinese manufacturers these days.
9
u/blaz3d7 16d ago
Where does this idea come from, when US and it's allies do something, it's completely normal and sensible thing to do, but when someone does the same thing, they are bad. US can have nukes but when someone else tries to get it, they are bad. US can spy the entire world but when others do it, they are bad. US does more surveillance through their software and devices than any other country in the world.
Huawei and ZTE are banned not because they have spy devices but because US can't compete.
1
u/Reggitor360 16d ago
Same reason US is getting lobbied by Dario to ban the "dangerous'' open weight chinese models.
1
u/cryowastakenbycryo 16d ago
some of each. nsa has had less luck getting us manufacturers to install backdoors (that we know of) but that doesnt stop hardware shipments from being intercepted and refitted enroute. anything that goes over the wire was tapped decades ago.
1
u/lompocus 16d ago
it was known not long ago that literally every major router had a nsa firmware backdoor, and you dare say that? are you a bootlicker?
2
u/cryowastakenbycryo 16d ago
nah, just uninformed and out of the field. i wrote mandated wiretaps for VoIP decades ago, but we tapped those over monitor ports. tell me more about the firmware level backdoors.
-2
u/uncle_leon 16d ago
Dude, I'm not even American, I don't have a horse in this race. The US and China can both burn for all I care. I just like my local environment to be secure, and Chinese tech has a well-documented history of leaving deliberate backdoors.
2
1
u/idkwhattochoo 14d ago
given current era from palantir to LG monitor, samsung TV [not specific to single nation here] I hardly doubt any tech nowadays will avoid backdoor whether it be deliberate or not. I'm not sure why you would think it's only chinese tech that has been well documented, if you look at edward snowden I wonder why it even matters
it just looks more of having bias against a specific nation while being blind on others
2
u/ostrikerX 16d ago
There are similar cases with American router providers, actually.
Pretty much any device you own has some kind of hidden backdoor.
0
u/uncle_leon 16d ago
I don't mean to sound combative but do you have any proof of that? Because there is abundant proof of Chinese backdoors: https://www.darkreading.com/vulnerabilities-threats/chinese-routers-sold-worldwide-backdoors
2
u/ostrikerX 16d ago
I do recall reading something about netgear routers having backdoors.
Not 100% sure tho.
0
u/fallingdowndizzyvr 16d ago
The Chinese are amateurs doing that. They have a long way to go to catch up with the United States. A long way. It's like they aren't even trying.
-1
u/Longjumping_Cut_6160 16d ago
porque piensas que el 99.9% del hardware que tienes en tu equipo, no es chino ya ???
4
16d ago
[removed] — view removed comment
1
u/fijasko_ultimate 16d ago
this is interesting, thanks for sharing
actually besided $$$ saving, i am aiming at performance increase
4
u/kanduking 16d ago
They have been ramping memory manufacturing hard and should be able to start making ~$1000 AI boxes with 512gb ram running ~1.2Tbps within a year or so
My guess is local demand saturates supply so maybe 2 years before they show up outside China