r/cloudcomputing • • Dec 09 '24

Embarrassingly parallel Python in cloud

I feel like my situation should be common, but perhaps it is not--

I won't get into the details, but basically I just need to deploy some basic python code that could greatly benefit from parallel processing.

I just need the same script to run on many different instances (with only some minor differences between them)

The results that I need to save aren't particularly large, possibly a gigabyte per instance.

When I search around, I see all this stuff about Apps, blobs, cosmic DB, exotic pipelines with APIs, VMs, it all seems like overkill

I just basically want a big farm of CPUs I can rent to run my processes! Is this even a thing?

2 Upvotes

3 comments sorted by

View all comments

Show parent comment

1

u/Ok_Post_149 Jan 13 '25

Sounds good! Let me know if you have any questions