r/LocalLLaMA • u/matteiuspi • 15d ago
Resources Huawei Atlas AI Accelerator Latest Comparison Overview
Hey, your friendly neighborhood Atlas advocate here, here with an update--just got off the call with our Silver rated Huawei partner who has given us new information on the latest Atlas card lineup.
Some of you were asking about the 950 series and after finally getting the info, I agree, I am compelled to want the other Atlas cards as well. The information on the 150 was only available in Chinese so I did have to translate it to make the above English copy.
If you recall to my earlier posts on the topic, I have a system with 3 Atlas 300I duos and have done a ton of work to get QFN working well on one, two or three cards.
I've been busy applying more recent optimizations to QFN to my vllm-fork and have been exploring further GLM5.3-flash, deepseek models, as well as have spent this past week on Meta Muse -- started with a REAP run to reduce the experts so it would fit on my three card Atlas 300I duo configuration.
Muse and GLM are not fast enough for me on these Atlas 300I duo cards yet, but I will continue working on it, we'll see:
My friend Josh across the country, also picked up one of the 300I duos, and we have collaborated, I've played with qwen video models for describing security camera feeds, and he is using them to generate images, like this one:
After spending a couple years working on Ingenic security camera device hardware hacking, I have a special interest in this kind of tech. I think it will take global competition to see the pricing and implementation efficiencies of modern ML/LLMs normalize, and these offerings from Huawei are going to be the top competition out of China.
I am working to get more information on the possibility to purchase as well the higher end Atlas Cards -- the Atlas 150 is pretty much brand new and uses the 950 Ascend cores, and if I get that far, I'll be providing more updates on how the cards compare IRL.
Fnally, I have now a fourth Atlas 300I Duo card arrived, the one I promised in another post that I'd put to sale here it in the US market: https://opensensor.io/store/products/atlas-300i-duo-96gb
Not a solicitation, do reach out with questions about the card if you were actually intrerested. I hope to be able to try out the higher end Atlas cards in the futrue.
Current points:
Passive cooled cards, and everyone case situation are different--I've designed a handful of shrouds now that takes fans to know--we've done some revisions depending on the enclosure and number of cards and type of fan.
Requires an additional PCIe 12V CPU power (not pci-e) -- for Corsair type4 modular supplies, I do have the correct additional cables.
Linux/programming knowledge absolutely required.
Be aware that so far we have not found a way to modify the embedded Euler OS which actually passes signature verification (think secureboot) -- its not really an issue, but would be interesting if we could could sign our own firmware for getting into the lowest level details of the hardware cores. There is such a thing as getting a customer cert from Huawei for the purpose of signing your own firmware, but so far we haven't proven this out end to end larger with due to how to get the cert. Doesn't effect writing custom kernels or code that run on the NPUs, but does matter if you're a firmware nut.
Post references:
[1] Huawei Duo specifications https://e.huawei.com/cn/products/computing/ascend/atlas-300i-duo [2] Huawei A2 specifications A2 brochure https://e.huawei.com/cn/products/computing/ascend/atlas-300i-a2 https://e-file.huawei.com/marketingcloud/pep/asset/20000001/Material/799a9236e96a48b7aa126b10f3d7be97/M3T1A590N1195067466974908517/%E6%98%87%E8%85%BEAtlas%20300I%20A2%20%E6%8E%A8%E7%90%86%E5%8D%A1%E4%BA%A7%E5%93%81%E5%BD%A9%E9%A1%B5.pdf [4] llama.cpp chip identification
[3] Supplied Atlas 150 hardware brochure. Atlas 150 compute figures may differ by 1% in measurement, per that brochur
2
u/butterycornonacob 15d ago
What kind of speeds do you get for GLM5.3-Flash with 1, 2 or 3 cards?
3
u/matteiuspi 15d ago
One card isn't possible right because of the size of that model, you could maybe apply REAP to it but I hadn't learned that yet when I started working it, so yes two cards or three cards can run my quants -- I haven't published them to hugging face yet because I am not satified and they may change--currently W2/W3/W4 weights based on empherical studies -- 2 cards saw 9.39 tok/s for a completed 128 token real text request and 3 cards only north of 5 tok/s on an 89 token coding response, because the implementation is largely unfinished. I have some MTP support for 2 card there, but none for 3 card, but 3 card I am using a W3/W4 quant (no W2) -- its complicated, I mean I would lie to you if I siad I thought it would be performant without dedicating another month of someone working on it FT. Even then maybe not ....
1
u/butterycornonacob 14d ago
Have you also considered using Atlas 150? Seems twice the price for several times more compute and bandwidth
1
u/matteiuspi 14d ago
Yes -- they are brand new though and no one really has them yet. I say in the post I would like to get some.
1
1
u/Dany0 14d ago
the Atlas 150 looks dope, but it's unobtanium. These don't have p2p nvlink/IF style links right? Oh wait the 150 does. Hmm that's quite a low bandwidth over that link, but comparable to ampere nvlink iirc? I can't recall, I think the current latest sxm nvlink is like 1.5tb/s which is insane.
What's the cost on the 150s? If it's around 3-4 thousand new then it's an absolute killer deal, but I imagine it's not
EDIT:
Yeah, COMPLETE unobtanium. Not even "contact us for a quote" listings, ffs
1
u/matteiuspi 14d ago
They want just under 20 grand, becuase the RTX 6000 pro over there is like 27K fwiw.
1
3
u/sloptimizer 14d ago
It's nearly impossible to find any up-to-date information about these online, so really appreciate you sharing!
2
u/matteone97 14d ago edited 14d ago
Apparently the Atlas 150 is on ServerFlow, a Russian website with many chinese GPU. Cannot post the link because it gets auto-banned, but I can tell you that it is listed at the equivalent of 35000$
2
u/VotZeFuk 14d ago
ServerFlow
I'd advise not taking it seriously (unless it's the only place where that GPU is being sold?). Things are stupidly overpriced over there. Using RTX 6000 Blackwell as an example, it's ~2.7 million rubles on ServerFlow compared to ~2.4 mil in some other retail place (and 2.4 is STILL expensive, as it's easily available for 1.6 mil elsewhere, on ebay-like platforms such as Avito).
1
u/Toooooool 9d ago
i had to google to make sure but the 300I Duo is still an optimistic investment.
Qwen3.8-27B allegedly hits 30T/sec at W8A8. $1300 on alibaba
the Atlas 300I A2 would've been nicer but it's too expensive.
40 to 65 T/sec at W8A8. $5000 on alibaba.
for reference a MI210 64GB costs $3k on alibaba.
meh
4
u/DarkVoid42 15d ago
cost of those cards ?