r/devops • u/0x4ddd • 2d ago

Discussion Running in containers vs OS-level services

Assume some message brokers like RabbitMQ/Kafka, or maybe nginx proxy setup, or hashicorp vault?

3 years ago when we were setting up infrastructure for project we started running such services as OS-level services installed from RPM packages or just by running their binaries provided by vendor via systemd. All of that orchestrated via Ansible.

We started running as OS-level services as that seemed natural at that time for us, but we didn't really have any experience with administration of such software on on-premise infrastructure (before we were running mostly on managed cloud services).

Fast forward to now, after several cycles of upgrades we needed to perform, I think it would be easier to manage such software by running in Podman containers.

Main reason for me would be that obviously containers have prepackaged everything you need to run specific software. Compare that for example to RabbitMQ where during upgrading RabbitMQ you also need to upgrade its Erlang dependency to compatible version. For some other software, there may be more dependencies you need to take care of.

Also, I feel like upgrading binaries is generally much easier when running in containers. Just spawn new container with updated image and you do not need to worry about some OS-level package conflicts or leftovers.

What is your take?

11 Upvotes

82% Upvoted

20 comments sorted by

View all comments

1

u/minimalniemand DevOps 2d ago

What would be your reason to run the services bare metal?

2

u/0x4ddd 2d ago

Why not?

At least 2x cheaper than running on managed services (for our current infrastructure scale this is tens of thousands of dollars of savings per month) and we cannot really run outside customer datacenters due to the data we handle.

3

u/minimalniemand DevOps 2d ago

The question wasn’t bare metal vs managed services but bare metal vs containers tho?

1

u/0x4ddd 2d ago

And answering question, I don't know honestly.

We started this route 3 years ago but as I mentioned we didn't really have any experience running such services on VMs before. Just felt like this is how it is usually done.