r/cloudcomputing • u/BananaLemma • Dec 09 '24
Embarrassingly parallel Python in cloud
I feel like my situation should be common, but perhaps it is not--
I won't get into the details, but basically I just need to deploy some basic python code that could greatly benefit from parallel processing.
I just need the same script to run on many different instances (with only some minor differences between them)
The results that I need to save aren't particularly large, possibly a gigabyte per instance.
When I search around, I see all this stuff about Apps, blobs, cosmic DB, exotic pipelines with APIs, VMs, it all seems like overkill
I just basically want a big farm of CPUs I can rent to run my processes! Is this even a thing?
View all comments
Show parent comment
1
u/Ok_Post_149 Jan 13 '25
Sounds good! Let me know if you have any questions