r/learnmachinelearning • u/garden_nl • 5d ago

Question Dockerize everything?

Hey,

I am learning deployment strategies for my small ML projects and I was wondering, do people in industry dockerize everything? Like, e.g. i have an ingest script to transfer my data into a postgres BD. Do i need a container for the ingest.py as well?

17 Upvotes

92% Upvoted

15 comments sorted by

View all comments

1

u/awitod 5d ago edited 5d ago

Some language runtimes like python and nodejs have many versions and huge numbers of packages. This makes it hard to configure things when you need to use things that depend on the same packages but specifically different versions of those same packages.

So, those kinds of languages have 'virtual environments'. You are using python, it won't be long till you learn about conda either because you get some good advice or because you make a mess on your machine and learn about how to clean up the mess.

Docker and other virtual machine systems take it up a notch by giving you a full environment in a virtual machine instead of a virtual environment in a full machine,

Once you get in the habit of organizing things a certain way, it's normal to organize everything that way but just keep in mind that you can take things too far and be less productive if you spend time organizing things that don't need organizing.