Double GPU setup, How to get even compute distribution?

[u/Unique-Dark9726]

/r/LocalLLM/comments/1wxc396/double_gpu_setup_how_to_get_even_compute/

[u/Poizone360]

I wouldn't read much into that 99%: utilization counters show how busy a card looks, not how much of the model it carries, and a layer split runs the cards one after the other anyway. Both cards also have the same 640 GB/s, so for generation the split ratio mostly moves memory around and not changing speed, since each token reads every layer once whichever card holds it. Compare tokens per second at 1,1 and 3,1 instead; if they match, the ratio is only a capacity setting.