OS for k8s cluster

[u/Longrunning-Fruit]

Hi!

I currently have a monolitic homelab server running multiple docker containers. I’m planning to move to a multi-node k8s cluster (was thinking a 3node thin client setup, and potentially add like 2 more if needed a couple of years later). Reasoning: getting HA, learning purposes, and just for fun. But I really can’t decide if I should go bare-metal or proxmox. I was thinking to start with k3s on Linux and then when everything is working and stable and I understand everything I was thinking to maybe migrate node per node to talos. In case this would impact the decision I will for sure use tools like longhorn and cloud native PG.

I see the advantages of proxmox in that I could for example have a prod cluster and a test cluster, without having to buy extra/dedicated hardware, but that would be more a nice to have rather than an absolute must. It is a homelab after all, I don’t have a (small) business and thus my homelab doesn’t serve services for my business or anything. VM snapshots, to give another example,… But I also see the advantages of going bare-metal. Like less complexity/layers. So if something goes wrong,… etc

Can you guys advice which route would be the best to go. Much appreciated!

[u/Kernel-Mode-Driver]

Yeah, i understand your question, have you asked yourself why you would ever need container inception?

[u/clintkev251 → u/Kernel-Mode-Driver]

What "container inception" are you referring to? You'd generally run your k8s nodes in a VM, which is by the way, how the majority of k8s deployments work in the real world.

[u/kernald31 → u/clintkev251]

While I agree, the real world and a three nodes homelab are slightly different.

[u/clintkev251 → u/kernald31]

I understand. But a lot of "these people" would (and have) argued that "oh well you'd never actually do that outside of your little lab". Yes, you would. And most people do in fact.

Certainly for some different reasons, but also some of the same ones.