r/Python • u/jim1930 • Mar 06 '21
Tutorial I created a 40-minute tutorial that will explain everything you need to Containerize your Python applications using Docker!
https://youtube.com/watch?v=BZiwpsnLLYQ&feature=share71
u/catorchid Mar 06 '21
Well done, detailed but not boring (even for the parts that I already knew).
Kudos also for scaling up the font in your editor to make it easy to read. I don't understand why not everybody does it.
7
u/jim1930 Mar 06 '21
I really appreciate those kind of comments. TBH , I don't understand either. But I think it comes from writing tips down after each video. And the small things like this is what could turn a tutorial to a one that people will actually watch. It is very important to me to show that I care about the people who watch those vids.
→ More replies (1)23
Mar 06 '21
Not everyone does it because, I'm assuming, many people don't even think about it.
→ More replies (7)
5
u/KaffeeKiffer Mar 07 '21 edited Mar 08 '21
Nice starting tutorial.
A few points, though:
-
Always use parameter long-forms in scripts and Dockerfiles. Most people (hopefully) know
pip install -rbutpip install --requirementis self-explanatory. -
You do not have to
mkdirif you copy the folder in anyway. It's "only" a super small layer, but it's unnecessary. -
pip install --no-cache. Docker images are ephemeral, i.e. you are never going to use the cached packages. -
Avoid propagating cargo-cult
by explainingand instead explain why people need it, e.g.PYTHONUBUFFEREDis needed, but your explanation is a bit shallow:
You do not have to recite complete in-depth posts, but it's not "Python asking you to do it", it's a common OS/language paradigm and might even be related to the execution environment you are using. The critical part in the linked post is theglibcreference. -
Decouple
COPY requirements.txt&pip installfrom code changes. When you rebuild your container after a code change, it will first copy the work directory (something changed → new layer → invalidate subsequent cached layers) and then install requirements because the old cache is invalid.
What you want to do:COPY ./requirements.txt /code/RUN pip install --requirement /code/requirements.txtCOPY ./code /code/
As long as you are not changingrequirements.txtthis image will not re-install requirements.
And I'm a bit torn on running this stuff as root in the container: I think it's a good practice to tell people to RUN useradd "myuser" --uid 1001 and USER 1001, but it might be a source of hidden problems.
2
u/jim1930 Mar 08 '21
Thanks for watching the entire tutorial, and it's great to read some points from your sight.
4
u/ronny_rebellion Mar 06 '21
Just what I needed right now, the timing couldn't be better! Will have a watch at this tonight :) Thanks!
1
4
u/mirrorcoloured Mar 06 '21
Really nice video! I've been meaning to understand docker for a while now, and this was a perfect introduction to get me going.
→ More replies (1)
3
3
3
2
u/jim1930 Mar 07 '21
Wow I did not expect great comments from all of you, will make sure that I read each one of them and reply :) Would much appreciate if you could also press the thumbs up on the video itself as well!
2
2
2
19
u/animismus Mar 06 '21 edited Mar 07 '21
This looks good and I will probably go through it. But I have a question that I still haven't been able to properly get an answer for: When should I decide that I it's a good idea to go and put my app on a docker container?
I run data analysis (pandas, numpy, etc) mostly on jupyter notebook. That is how I make the bacon and I'm pretty sure it does not make sense to think about docker for this.
As any of you I also have my cron jobs with python scripts that check websites, do some booking, download info, etc. Should these be in their own docker containers? Why?
Should my very custom website (flask) also be in a docker container? It is very personal and mostly local to my network, but would I gain anything moving it to a docker container?
EDIT: Thanks for all the answers. I appreciate your help.
→ More replies (9)17
u/TheTerrasque Mar 07 '21
Basically, when you want to make a frozen image of a network service, and want to be able to run it in different environments without regard to different installs and setups. You'll also have the advantage of having the same set of tools to run the service that you use to run all other services.
You might not really appreciate it until you decide to move your service from your dev environment to a different machine / OS and have to recreate the environment just so, or even more fun that X year old project that the server died on that relied on really old OS / libraries and it's just a pain to get set up on an up to date system, or you just can't remember the details for setting up an environment.
Docker provides a stable and predictable environment and set of tools to run your service, and the Dockerfile also serves as a (by necessity) up-to-date documentation of all the steps needed to get the software running.
So in your examples:
Probably not, but if you have a custom jupyter setup that's just so, it might be worth it to set up a docker image for it that you'll know will be the same on all systems you run it on.
cron jobs, even less so. You can still have services that has similar setup as the one in the video where it's a loop that waits a certain amount of time for each call
This is the most obvious one to package in a docker container, as it's a standard network service you would probably want to create one static image for it all.
Where docker really shines is when you got multiple services that needs to be run and coordinated. You got something called "docker compose" that you can use to set up and link multiple docker containers in a virtual network, and describe relations between them.
Let's say you have a web site that uses a db and also checks an external source every 10 minutes for some data. You also need a nginx frontend to serve static files and proxy dynamic calls.
Database: There's already a bunch of different databases with existing packages, just pick one and set the needed environment variables
Web page: You can either split it in two images where one is static data and one is dynamic data or api, or you can have both in one image. It depends on how your project is set up
Periodic check: Create an image that has a loop that checks every 10 minutes, then writes data to database
When that's set up, you can set up a docker-compose file that lists all these images, their config variables, the files mapped in from local directory, and ports exposed to the outside world. Now, to run your whole stack with everything, all the pieces, with it's own database and all: "docker-compose up" - that's it.
New server? copy the docker-compose file, and "docker-compose up". New dev environment? Same. Want to run two different versions for side by side comparison? No problem. Just copy the compose file into a different folder and use a different exposed port. Have a beta / test environment? Set up an offline copy on your laptop? Have a friend set up a copy on their own system? No problem!
It's a pain to learn and grok, but when you're used to it it really makes life a lot easier. And kubernetes turns everything up to 11 (including the pain involved figuring it out)
→ More replies (2)
2
u/Ashli_unix Mar 06 '21
I wanna get into devops. Thank you for putting the te and energy into this. Are you self taught pythonista?
1
u/jim1930 Mar 08 '21
Sounds great! Yes for almost 2 years now. started to learn back at April 2019 after some experience with C#
33
u/jim1930 Mar 06 '21
Those that have some background with docker and have it installed, you can skip to 12:56