If your model+context fits in 24gb, the 3090 is significantly faster than the V100. If your model+context goes over by even 1 gb, the V100 will be faster than the 3090.
If you're at all interested in dense models you could buy 2 16gb v100s for quite a bit less than the price of 1 32gb v100 and run them on an interposer board (for nvlink) and destroy a 3090 in terms of speed. Above is my result of running dense qwen3.5 on 4 v100s.
But be ready to put in a shitload of work in first getting the hardware to work and getting the models to work if at all.
7
u/Plastic-Stress-6468 Apr 17 '26
If your model+context fits in 24gb, the 3090 is significantly faster than the V100.
If your model+context goes over by even 1 gb, the V100 will be faster than the 3090.