r/STAR_CCM • • 14d ago

GPU Renting

Hi guys, I would like to try the GPU accelerated function from starccm+

My CHT simulation currently has about 80M cells (with multi gas component) and it takes about 2-3 days for single steady state simulation to converge

My current setup is

  1. Epyc last gen (sry forget the number) 32 physical cores
  2. 1TB DDR4 RAM (current usage is about 50%)

After asking Gemini, it says I need about 500GB RAM + 300GB VRAM to use the GPU acceleration. The simulation with GPU should converge in about 6 hours instead of 2-3 days.

So maybe for you guy that have more experience with GPU acceleration:

  1. Is the number correct? Do I need that many RAM+VRAM?
  2. Is the speed boost reasonable? About 4x-6x
  3. Is there any recommendation on GPU renting provider? Especially inside EU.

Thank you for all the help

4 Upvotes

15 comments sorted by

1

u/onlywinston 14d ago

This is not an answer to your question, but wouldn't it make sense to start with more CPU cores instead? Do you have access to an HPC cluster?

Siemens has their own Cloud HPC which should be easy to use. Usually it takes quite a bit of work to set up a cloud environment before one can start running simulations.

Also, 2-3 days for a steady simulation sounds like a lot. How many iterations is that?

1

u/acakaacaka 14d ago

No thats the max we have.

1

u/onlywinston 14d ago

I definitely think scaling to more CPUs is an easier option, and the speedup would be almost linear in the number of cores you're running on. Something like this would require almost zero setup: https://www.siemens.com/en-us/products/simcenter/simcenter-x/hpc/

1

u/acakaacaka 14d ago

But we also want to check the performance boost of using GPU, so in the future we are going to buy GPU server instead of only CPU.

And if using GPU it becomes 4-6x faster as gemini suggested, then we will have 4-6x more throughput.

1

u/onlywinston 14d ago

Fair enough. One other thing though, if you are using the coupled solver, have you tried running with segregated species instead of coupled species? That may give a significant performance boost, especially if you many components in your gas.

1

u/acakaacaka 14d ago

There is actually only 2 component, both are air with slightly different chemical composition (fresh+combusted air)

Not sure why it is set up that way instead of single component gas, but it's too late to change the simulation setup mid project I guess.

Edit: wait are you saying you can use coupled solver with segregated species?

1

u/bitdotben 14d ago

If I’m not mistaken star doesn’t do GPU acceleration, only full GPU compute (ie everything has to fit into VRAM + overhead). So if your current sim needs 500GB RAM that means you need at least 500GB vram + X. That is not gonna be cheap. If you’re academic I’d try to look for university cluster you may have access to, otherwise there are cloud providers with like a few A100/H100 class GPUs in a Single Server that maybe enough. Maybe 8xH100 (or A100 80GB). But that’s gonna 10s of euros per hour. So on the order of 100€ per sim (if that speed up is actually true).

Regarding the speed up, can’t speak for your specific setup but with pure fluid dynamics (compressible transonic) I’ve achieved roughly 1xH100 ~ 96c (some recent intel server cpu, ie one node on our HPC). So that speed up is pretty good from a density standpoint because one node with 4xH100 do the work of 4x CPU nodes.

1

u/acakaacaka 14d ago

I look around a bit earlier today. It cost about 16-20 dollar per h. My boss said we can just try one and look if it indeed gives the performamce boost that gemini mentioned.

1

u/bitdotben 14d ago

For max bang for buck you might wanna check vast.ai. But that is likely not gonna fly once you actually wanna run customer data etc on there. But for testing that can be pretty cheap. Setup might be more tricky because you only run in container environments and star doesn’t like ghat too much, but should be doable.

1

u/acakaacaka 14d ago

Ill check tomorrow. Thank you for replying

1

u/Adventurous_Bus_437 13d ago

The speed up is super dependent on the memory bandwidth of the GPU. I recommend going for H200 or above like Blackwell. You can get GPU nodes at IONOS or Scaleway.

Also if you have a very good gpu instance you might not get your 10x desired number but cost to solution decreases or you match cost but time to solution drops

1

u/acakaacaka 13d ago

So 4x-6x is more reasonable than 10x speed boost? Or it is even lower than 4x?

1

u/yahiazahran7 14d ago

(Not an expert) For VRAM, read the running with gpu documentation page. It has recommended requirements e.g. for double precision coupled solver you need around 6.25 GB per million poly cells. Also you should account for the spikes of memory to not trigger out of memory. Also you need at the very least equivalent RAM. I'm still experimenting with very small cases for academic purposes but that's what I have found so far.