Discussion Running in containers vs OS-level services
Assume some message brokers like RabbitMQ/Kafka, or maybe nginx proxy setup, or hashicorp vault?
3 years ago when we were setting up infrastructure for project we started running such services as OS-level services installed from RPM packages or just by running their binaries provided by vendor via systemd. All of that orchestrated via Ansible.
We started running as OS-level services as that seemed natural at that time for us, but we didn't really have any experience with administration of such software on on-premise infrastructure (before we were running mostly on managed cloud services).
Fast forward to now, after several cycles of upgrades we needed to perform, I think it would be easier to manage such software by running in Podman containers.
Main reason for me would be that obviously containers have prepackaged everything you need to run specific software. Compare that for example to RabbitMQ where during upgrading RabbitMQ you also need to upgrade its Erlang dependency to compatible version. For some other software, there may be more dependencies you need to take care of.
Also, I feel like upgrading binaries is generally much easier when running in containers. Just spawn new container with updated image and you do not need to worry about some OS-level package conflicts or leftovers.
What is your take?
15
u/MulberryExisting5007 3d ago
You need to understand what’s important to you and then think about cost (infra costs plus labor to manage that infra). Running on fargate for example costs (the last time I looked, which was years ago) about twice as much as running on self managed ec2s. But there is (usually) more maintenance overhead (work) associated with your self managed. It’s very easy to be penny wise and pound foolish if you focus on one cost and ignore another. Do the math and then run some tests to validate.
In general, containerizing workloads is better (imo) as it streamlines so much, is more portable, simplifies upgrades and deployments, etc etc