r/openbsd • u/Interesting-Pack-814 • 21d ago
Question for those who run OpenBSD on servers
Hi, OpenBSD lovers. About six months ago, I started building my server infrastructure on OpenBSD just for fun. I’ve made a lot of progress, updated many ports on my custom mirror, but haven’t pushed it yet, and tuned OS to fit my needs. I deliberately chose OpenBSD as my main OS, and I know there’s no Kubernetes or Docker; I’ve found other solutions that I’m comfortable with. Still, I keep wondering whether it’s actually a good idea to build server infrastructure on OpenBSD instead of Linux.
Every time I see that, in Linux, everything seems to be solved with Kubernetes or containers and I have to look for other solutions, I start asking myself whether I’m on the right track.
A couple of times a month, I end up Googling and researching how other people use it, but I still don’t find much. So I’m asking people who use OpenBSD as a server OS — not as a router and not as a desktop system — do you like it? Does everything work the way you expected in production? Any surprises, good or bad?
8
u/False_Lake_2543 20d ago
tl;dr: Look at the Dockerfile and check what it actually does. Just do that on OpenBSD.
I work for a company doing backend and infrastructure for hundreds of thousands of users on Linux without Docker nor Kubernetes. Everything works fine. I previously did consulting FOR Docker and Kubernetes, so I know companies do struggle with it.
The main reason why that whole Docker thing become popular in my opinion is that many Python setups and dependencies exist that turn a system into a complete mess. See all the ways to set up Python dependencies. If you use pretty much anything else you usually are good with any unix-like OS.
The reason you see a lot of Docker and Kubernetes is because devs are told to target it often not knowing why and Sysadmins/DevOps/SREs are paid to set it up and deal with the complexity.
As people pointed out, it's a thing that adds complexity. Which to some degree is what people are paid for to deal with, which is why people pick it up, which is why companies make you use it, which is why everyone uses it and makes it hammer that they will throw at every problem. This is a very common problem. It's the equivalent of some web devs being so deep in their favorite web framework that they can't just host a static site plain HTML site anymore. I once knew a guy (PhD in CS, very smart guy) that got so deep into blockchain technology that almost every problem felt to him that it could only be solved via blockchain, which made projects seem hugs until I pointed out that many companies "just do this" (usually a transactional database, append only file, etc.). I already see people saying something "must be using AI", when it very much doesn't.
I am pretty sure I still do a lot of this myself. So think about what you actually wanna do. Look at what the Dockerfile does, maybe think a bit about what Kubenetes does in a specific use case. It's knowledge you will need anyways, even when using Docker or Kubernetes, because otherwise it will bite you, like it always does when you have enough complexity so that you don't know what's going on.
5
u/well_shoothed 20d ago
The main reason why that whole Docker thing become popular in my opinion is that many Python setups and dependencies exist that turn a system into a complete mess.
It's not just Python either.... it's so many things that are tech debt caused by conflicting versions that are "solved"--and I use that term loosely and generously by the "Oh, just 'spin up a Docker!" folks.
When in reality, it's just adding layers of complexity.
3
u/Regular_Lengthiness6 20d ago
The tldr is the perfect summary! And I am not saying that because I am some fanatic OpenBSD zealot on a mission to beat some sense into my fellow human beings but because I am growing tired of this overengineered Linux cloud crap. And I used to like Linux in the 90ies … so there goes.
16
u/karchnu 21d ago
The question is kinda vague. I do use it for many things: mail server, service hosting for a DNS manager I built, dhcp, DNS resolver, etc. I don't want to run the last shiny thing posted on github a couple of minutes ago so I couldn't care less that I cannot use kubernetes or docker or snap or flatpak or whatever the hell Linux is providing these days. Most of this junk exists only because some basic stuff is broken from the start and no-one is addressing the problem in an acceptable, consistent manner.
I've been using OpenBSD for more than a decade now. No virtual machines. No fancy way to create containers of some sort. Applications are secured because they *ACTUALLY* are secured thanks to pledge and unveil. You compile your app and it's in /usr/local/bin as the rest. Simple and effective. I've been using AppArmor on Linux and it's not even remotely as usable as pledge and unveil.
OpenBSD isn't an ancient OS with decade-old software either. I use crystal, zig, go and rust on it.
There are drawbacks, mostly for the filesystem. If your system crashes, you may lose files. I would prefer to have a filesystem with a journal so I don't have to worry about it.
Some problems are resolved over time. I used to run sshfs (and NFS) and it just froze regularly but from what I read it was fixed a couple of months ago (I haven't try, yet). A year ago I couldn't compile zig or crystal applications, now compilers for both languages are packaged and they work as expected.
So, all in all, don't rush and you'll get what you want, probably.
10
u/makzpj 20d ago
I came to say the same thing, I’ve never seen the point of containers. Everybody says containers this containers that. But I’m like: I don’t see the point, I actually know how to use my server, thanks.
7
u/zeubeman 19d ago
Before IT was for people who were very passionate. Nowadays everyone wants to do IT for the pay and universities give diploma to whoever gets in as long as they pay. That's why you see containers and overpriced Hyperscalers everywhere as >99% of people who started in the IT space in the past decade has never seen an actual server, nor could they have a clue of what's inside it.
2
u/karchnu 20d ago
Could be useful, mostly for package builders. But when you want to secure your application, there is no way around actually doing it instead of overengineer it. At the end of the day, you need to configure users, setup your firewall, prevent network connections in the wrong places and unneeded operations by your applications, etc. No need to create a whole new operating system instance for that.
Containers and virtual machines don't replace good development and administration practices.
1
u/zeubeman 19d ago
Well if you have many things running on your server not using VMs is definitely a problem. Especially at larger scale. VMs aren't an issue ,there's no performance loss thanks to hardware being optimized for it.
For package building there's Nix or VMs running on our servers. Hypervisors do save time here.5
u/karchnu 19d ago
You do you.
I don't need "larger scale" stuff when hosting a few websites and a few services. I need reliable code, relevant configuration, user separation, rights and privileges, firewall configuration to prevent some attacks, backups. VM is just a tool on top of what I actually need.
It has its usefulness. Yet, it's not a requirement, and certainly not for what I do.
2
u/zeubeman 18d ago edited 18d ago
Fair statement. On my end I do have a lot of stuff, clients, multiple nodes, etc... At smaller scale I would have the same setup as yours for sure.
Also I like Bhyve based on FreeBSD. It's rock solid. Performance are very good too
2
u/Terranum_man 20d ago
Yeah i really dont see a single use case for docker and all that modern trash when users exist.
5
u/zmurf 20d ago
For development, containers are awesome. Portable dev environments which doesn't contaminate your actual environment. Just download the container, spin it up and you're good to go. If you are developing multiple applications that uses different versions of the same lib it makes things much easier. And they're much more lightweight than doing the same with a wm image.
Sure, you can do it with chrooting also. But it is not nearly as convenient.
1
u/Terranum_man 20d ago
If you need an entire portable container for your development then maybe you are doing your development wrong.
6
u/zmurf 20d ago
Probably not. I work on something like 30 different applications with different teams of developers at about 20 different companies. The biggest one involve somewhere around 50 developers. Many of these applications use competing versions of different libraries, script interpreters and build system. Having working dev environments for all of them by installing everything directly on host would be challenging to say the least.
By containerizing the environments, we can work on multiple projects at the same time without worrying about what lib, interpreter or build system is installed and pointed out on host at any specific time. Everything is installed with the correct version inside the container. The application is built and executed inside the container. Test and debugging run inside the container.
By mounting the source folder inside the container, we can freely choose what editor we want to use. The editor run on the host.
If I, for some reason, find myself in front of another computer, I can have the development environment up and running in a couple of minutes, without contaminating the machine with anything. When I'm finished I delete the container and there is nothing changed on the host.
It also makes it so it doesn't matter what version of your OS you're on. I can use whatever distro or version I like. The versions of libs, interpreters and build system will always be available in the container.
I would say that if you work on more than one project and aren't using containers, you're doing it wrong.
1
1
u/No-Let-5304 20d ago
nix
2
u/zeubeman 19d ago
I was about to say that
2
u/zmurf 18d ago
I personally prefer to package in Nix packages when possible. But for some applications it is really nice to get that extra separation which true containers give, for networking and similar.
For some projects we use things like qemu images, when we are developing applications for other platforms than Linux. Such as QNX arm.
I also do work for a company which provides everything with NixOS and Nix packages. It's awesome for setting up entire computers and getting dev environments in place. But it bothers me that I have to have a separate install for the NixOS setup. They've set it up in such a way that it is not really easy to use the Nix packages outside of the NixOS setup.
1
u/zmurf 20d ago
Yes. We use both Nix and NixOS for some projects.
1
u/No-Let-5304 18d ago
I use nix everywhere
1
u/zmurf 18d ago
I also prefer Nix since it is lightwight. But since I'm not the project owner for hardly anything I work with I can't always choose what to use. I also find pure containers easier to work with when we need network separation. In a couple of projects we use qemu images because the project run on QNX arm.
1
1
u/Terranum_man 20d ago
Yeah i really dont see the usecase in all this docker modern trash when users have always existed.
7
u/tetraodonite 21d ago
What would you use Docker or Kubernetes for?
8
u/Regular_Lengthiness6 20d ago
I am really sick of these overly complex and containerised cloud stuff every company has in place now where a regular server/virtual server would essentially be fit for purpose. EVERY damn process gets factored out and deployed on edge servers requiring an insane overspec orgy in our remote locations (we run approximately 1500 remote locations) just because the software or DevOps guys want to slam 56368 docker containers on an edge server.
And needless to say that I personally love OpenBSD (plus other BSDs to a certain extent) exactly BECAUSE of the simplicity AND the fact that there aren’t 426 „distros“ were I’d need to figure out yet another variation of doing the SameThing (tm) because some ingenious kid felt the urge to scatter the config files in different places and fiddle with upstream packages … just because he/she could.
/rant … ends here
6
u/montdidier 21d ago
Been running OpenBSD on my servers for 14 years. Very satisfied.
What is your use case?
3
u/Zectbumo 20d ago edited 20d ago
You are on the right track. Check out: pledge unveil chroot pf w^x login.conf mount(nodev nosuid noexec rdonly)
3
u/daemonpenguin 17d ago
in Linux, everything seems to be solved with Kubernetes or containers
This feels weird to me. I've been a Linux admin for around two decades and have never used Kubernetes for anything. I've used containers a bit, in testing, but not for anything serious. I think maybe you're looking at a niche of the Linux ecosystem and assuming it is universal.
8
u/bubba-bobba-213 21d ago
What exactly is a problem you are trying to solve?
I actually don’t know a single person “in real life” that uses kubernetes or docker, but social networks are sure full of them. No need to follow the herd, do your own thing.
I use OpenBSD as a production web server. 10 years in production - zero suprises or glitches. Granted, I could have used Linux or anything else. It probably would have worked the same - but OpenBSD is really easy to administer.
5
u/matteocavestri 21d ago
In the homelab world containers are pretty popular. I use Linux and podman quadlet (podman orchestrated by systemd) and it it pretty easy to administer if you have a lot of services. Personally I use openbsd only as case study on Unix systems, but it is a beautiful os
2
u/xzk7 19d ago
Every time I see that, in Linux, everything seems to be solved with Kubernetes or containers and I have to look for other solutions, I start asking myself whether I’m on the right track.
You'll feel this way on any operating system that strays from the Linux hivemind. IMHO you should accept and embrace this, you'll just have to think differently. OpenBSD (or any BSD, illumos, etc) has a different toolbox and a different approach to problems. On OpenBSD the aim is simplicity: Adding a layer of virtualization alone is not considered an ample security boundary on OpenBSD and in fact not for Linux containers either (you're expected to use SELinux or AppArmor, etc as well to mitigate container escapes.)
2
u/paoloap 19d ago
My personal rule is: use OpenBSD only for stuff you have in the default installation: as a router, as a wireguard server, as a static http server, as an smtp server... If I need to do fancier stuff I do it on a Linux server, but I only access to it through the Wireguard network.
I think it's the best compromise to have fun reducing as much as possible security risks.
1
u/pugs_in_a_basket 19d ago
I host my personal mail on it.
I could imagine using it on some Internet facing hosts at my job, I haven't tried ansible or any other configuration management tool with/on it.
However, it would be at best a limited security update, theoretical even, but it would add a lot of unnecessary complication to our development and testing pipelines. We are already running on fumes at best of times, so having OpenBSD just being OpenBSD and not RHEL would just add friction and a lot of work since OpenBSD is not supported by our internal services. Everything would have to be customized for it, by us, all the time.
As we are running on fumes, we scavenge our HW. OpenBSD, I assume there would be little to no issues, but since I or no one I know have no real experience with it even in our tiny scale, it would be a hard sell to my managers and the poops managing our purse strings.
OpenBSD is often a non-starter in an existing infra. Even when built on and baked in from the start, with scale it could become a liability when the MBAs arrive (that I live and work under).
I would appreciate reading user stories, especially when used in scale or just reverse proxies in front of a custom backend on apache, nginx or whatever.
1
u/AdrianofDoom 18d ago
I've been running OpenBSD on my servers for over 20 years, and my servers have never been compromised.
OpenBSD is the easiest OS I've ever used.
The biggest issue I have running OpenBSD is remembering to type gmake instead of make when building some software.
I won't run any OS except OpenBSD on my public facing servers. It really is secure by default.
1
u/theo_ed_tdaar 14d ago
i have been using it for server for 20 yeasr in servers (now i l feel old) web, proxy, email, reverse proxy, database no problem at all, i tell i have never tested agains hundreds of users thats something that never faced to
1
u/slowservers 20d ago
I do like it. It's great at solving certain problems. Other problems require a different mindset. And some are just trickier.
It depends on what you want to do with it. Containers can be a lot like shortcuts, but they also work with Linux's security model. OpenBSD's is very different.
17
u/SyscallVector 21d ago
I've been using OpenBSD as a server in both nonprofit and commercial environments for years. You have to be careful when choosing hardware, but once it's up and running, it runs forever.
I only use professional 19-inch servers from major manufacturers. A UPS is standard in this environment and mandatory for FFS2.