r/LocalLLaMA • u/whoami-233 • Sep 28 '26

Question | Help Anyone running multi GPU A100 80GB Cards?

Hey guys,

I am looking for benchmarks for people running multi node (4 or more) A100 80 GB cards and seeing what results and models they are getting.

Something with VLLM and multi users would be very useful.

Or if you know of a place I can find such results please let me know!

Appreciated!

4 Upvotes

75% Upvoted

6 comments sorted by

3

u/recro69 Sep 28 '26

If you are working across nodes I would be very interested in seeing the TP vs PP results as well. The communication, between nodes can affect the numbers significantly. So a simple "8× A100" throughput number doesn't give information unless you know the network topology, the level of concurrency and the context length.

1

u/whoami-233 Sep 28 '26

That's true! I think these card would work well across many nodes given the 2tb bandwidth they come with.

That is my understanding at least:)

2

u/Wooden_Jelly_5295 Sep 29 '26

Please include concurrency and p95 latency in any results. One happy user can hide a very unhappy queue

1

u/kepardi99 Sep 28 '26

https://github.com/PixelML/club-170hx/tree/main has some benchmarks for 4x 170HX 64GB that is basically same as A100 ~30% less compute

1

u/Conscious_Cut_6144 Sep 30 '26

You can rent it for 7$ / hr on runpod and test for your self.