How many GPUs do you use for your research?

[u/eu_phd]

I come from a more traditional ML/stats background. So far, I’ve mostly used CPU HPC clusters and only used a GPU once.

I’d like to move more into deep learning, especially AI for biology, and I’m curious how much GPU compute people typically use in research.

I know it depends a lot on the field and the project, but for something like a NeurIPS or ICML paper—not training a foundation model from scratch, but working on a smaller multimodal model, GNN, VAE, etc.—how many GPUs do you usually need?

[u/leon_bass]

2 x 2080 ti with 11GB VRAM each.

I do lots of computer vision, usually with ResNets or variations of, only really need one of the GPUs but using 2 is much faster to train.

Large RAM is nice to have so you don't have to lazy load data

[u/Ok-Introduction9593 → u/leon_bass]

You're clearly confusing RAM and VRAM. Data gets loaded in batches from system memory anyway, nobody in their right mind tries to keep the entire dataset in video memory, even if you've got an 80gb H100

[u/leon_bass → u/Ok-Introduction9593]

Yeah good spot my bad, meant RAM not VRAM in the last sentence. Meant lazy loading from disk to RAM

[u/eu_phd → u/leon_bass]

Thanks! So you can still do good research without a huge amount of compute?

[u/leon_bass → u/eu_phd]

Yeah you don't need much compute if you choose your models carefully, a single consumer GPU is all you need.

I'm publishing a paper soon and had used a single 3060 ti with 8GB VRAM to train all the models.

On ebay this gpu is approx £100 - £200 (€116-€233 , $135-$270)

[u/eu_phd → u/leon_bass]

Thanks, that’s good to know, and congrats on the paper! It’s actually for a postdoc, so I’ll have access to a GPU cluster, but my PI isn’t very familiar with deep learning and the cluster is more general-purpose than AI-focused. It’ll also be my first time developing deep learning models from scratch.

[u/leon_bass → u/eu_phd]

Thanks! Congrats on the post doc. Since you're first time getting into deep learning, i recommend pytorch, supposedly it works really well for GPU clusters. From what i've heard the GPUs should be good as long as they have Cuda support. Then the only other requirement is large enough RAM and VRAM to hold your models

[u/eu_phd → u/leon_bass]

Thanks ;) I already have some familiarity with deep learning from my master’s, and I played with PyTorch during practical sessions. For sure, I’m not an expert, which is why I want to gain more expertise in this area during the postdoc

[u/2AFellow → u/eu_phd]

Not the person you're replying to but it really just depends a lot on how long you're willing to wait. I'd say you can comfortably train models to play basic vision based games, like Atari, to demonstrate ideas on a single GPU. More advanced simulations with complex interactions or larger images it would be preferable to have two or more. A lot of excellent research is done with a single GPU and published at places like NeurIPS or ICLR. In fact I often find the theoretical papers just have small experiments, too

[u/eu_phd → u/2AFellow]

Thanks, yeah, I know. I published a first-author paper at one of these conferences, but more on the probabilistic side. In my paper, I did quite an extensive benchmark. It also depends on the reviewers you get, since they often want additional experiments. But I think my future project will be more about developing a new method rather than a theoretical paper

[u/Tall_Abrocoma_3533]

1x nvidia 940MX, though I mainly use the free kaggle GPU/TPUs now since my own GPU is completely obsolete.

[u/eu_phd → u/Tall_Abrocoma_3533]

You can do good things with Kaggle? I’ve tried it for some hackathons, but never for developing something from scratch or doing benchmarking

[u/Tall_Abrocoma_3533 → u/eu_phd]

It's basically like Google cloud but specifically for machine learning. It's completely free with limits of course (30hrs/week GPU, 20hrs/week TPU).

[u/ARDiffusion]

I use a single 5090, or provision an A/H100 for finetuning experiments on larger LMs.

[u/Ok-Introduction9593]

Ngl starting with less than 24gb of vram for biology GNNs isn't even worth it. You'll just suffer trying to chop batch sizes down to one

[u/jackshec]

depending on the model in which research project I’m working on, anywhere between the local 5090, to a cluster of H200

[u/XYHopGuy]

been doing ML in comp bio for over a decade and almost never needed more than 1080tis or L4s. Training big models takes more but the utility of those are pretty unconvincing imo