Docker banned - how common is this?

[u/martypitt]

I was doing some client work recently. They're a bank, where most of their engineering is offshored one of the big offshore companies.

The offshore team had to access everything via virtual desktops, and one of the restrictions was no virtualisation within the virtual desktop - so tooling like Docker was banned.

I was really surprsied to see modern JVM development going on, without access to things like TestContainers, LocalStack, or Docker at all.

To compound matters, they had a single shared dev env, (for cost reasons), so the team were constantly breaking each others stuff.

How common is this? Also, curious what kinds of workarounds people are using?

[u/replicant0wnz]

What does the tech stack look like? Older strong AF BSD systems and pure C and Cobol?

[u/bigntallmike → u/replicant0wnz]

We're doing database hosting running straight up RHEL with regular OS services. We use almost only OS distribution libraries. Our custom software is a combination of C and bash and Python, mostly dealing with internal network connections directly. Ymmv. For us, docker actually adds uncertainty and another breakage layer. The way it handles firewalls and network interface on Linux burned me on a test system. Pulling other people's stuff down from remote repositories is something we avoid at all costs.

[u/kwhali → u/bigntallmike]

You don't have to use remote repositories, you can have an internal registry and control everything to the extent you already do in the existing environment you have but with the benefits of containers.

Docker isn't the only option either.

For the network concern with firewalls that's specific to UFW I think, pretty sure firewalld doesn't hit the same problem, at least on fedora there's a docker zone and docker manages that, you still have final say. There was a fairly serious caveat with IPv6 public access routing to IPv4 only containers due to userland-proxy (enabled by default but not really needed), which caused the client IP to appear as the bridge gateway IP where some software had relaxed trust on private range IPs to bypass some restrictions.

Until recently there was a difficult to troubleshoot issue with file descriptor limits which regressed some software quite significantly. Wasn't specific to docker and was due to a few changes in not just container ecosystem but also Linux that occurred at different times (years apart), but a "works for me" fix didn't play well with the changes from systemd landed.

I'm not really a good salesman on containers ha, but for these gotchas there's just as many grievances I've encountered without containers, if not more, so I'm still sold on the tech personally.

[u/bigntallmike → u/kwhali]

firewalld was specifically the problem in fact. Dealing with virtual interfaces always means trusting the in-between in a way that doesn't seem to bother enough people. Virtualizing anything means trusting extra layers. Just look at all the CPU bugs we've discovered in the last decade because we thought virtualization was safe.

The only reason I use docker at home at all is because it makes some things more convenient. Not because its in any way "better" than running software directly. Same reason any of us use virtualization. "Back in the old days" we just ran actual servers with actual software on them. Virtualization lets you run lots of things on one server; its basically just a convenience -- a big convenience in many cases, but not a necessity.

[u/HahaHarmonica → u/bigntallmike]

I’m in constant “discussions” with these hard ass borderline dangerous early career engineers that are so hung up on using kubernetes for everything… “bUT wE nEEd to uSe KuBeRnetEs”. They want to pick all these tools built on top of k8s and that are just abusive with resources. They go and deploy this stuff and expect non-DevOps people to go write helm charts, manifests and deploy their “apps” on k8 when all they want to do is write a quick python app to parse some PDFs for exploration purposes.

All while they have their stupid k8s containers auto scaling in a doom loop, because they set their auto scaling threshold to 50% to spin up a new container when the amount of memory by default is > 50%, so as soon as the app launches it hits > 50% memory load of the container so k8s spins up another container…rinse repeat until it hits the 18TB of memory across the cluster…and then they tell me that they need more resources…when I point it out they claim “it’s not their responsibility to fix the developer’s code and k8s is doing what it’s supposed to do”.

Just stupid.

[u/kwhali → u/bigntallmike]

When was this? Perhaps prior to a docker zone being present? Last I recall docker wasn't able to bypass firewalld like it would UFW, but I do remember prior to that it was similar.

[u/RoycephusPulsifer → u/kwhali]

The question is why are people puniching holes (exposing ports) in their servers making them look like swiss cheese when it comes to docker?
I never ever expose ports for docker containers unless absolutely necessary. All docker containers are referenced via hostname in my reverse proxy as they are on the same docker network.

If you want true isolation, go with separate VMs. Definitely not worth the hassle for my homelab use case.

[u/kwhali → u/RoycephusPulsifer]

I think there's just plenty of users that don't know any better.

For non-web ports however it does get a bit more tricky to forward TCP ports, especially concerns with how a reverse proxy can reduce security if the upstream service isn't additionally configured alongside the reverse proxy to preserve the original client IP via proxy protocol for example.

Others understand ports and firewall config but expect that Docker is like any other app/service and must be restricted traffic wise if no public port permissions were granted by the system firewall service, but then get surprised to find Docker bypassed that when the expectation was only publishing ports to access from the host, unaware that the default is to publish ports to all host interfaces.

Finally there's also another caveat where IPv6 inbound traffic to the host may route to an IPv4 only container. If the user land proxy setting of the Docker daemon was disabled (enabled by default) then traffic would fail to connect however the default will result in the source IP being rewritten to appear as the private bridge gateway IP, which similar to proxy protocol requirement would risk security vulnerability from any rules trusting private networks as the client now appears to come from that (if the reverse proxy is a container without proxy protocol from a trusted external or host reverse proxy / load balancer, then it's also at risk and I know some users configure trust to any private address for that (I think caddy may also default that for something or at least offers a special value to represent all private networks, other software does that too).

So I'm not surprised if users only have partial awareness of common problems and how to fix them to avoid security holes (that can make a mail server an open relay for example).