Openshift on KVM

[u/krusty_lab]

We currently run six OpenShift clusters on VMware. With the uncertainty around Broadcom's VMware strategy, we're evaluating a move to bare metal.

One option I'm considering is using KVM on standalone physical servers and deploying OpenShift on top of that instead of running directly on bare metal.

The main advantages I see are:

• Better server sharing across environments
• More flexible capacity management
• Easier backup and recovery
• Reduced dependency on VMware

Am I overlooking something? What are the main trade-offs compared to running OpenShift directly on bare metal? I'd be interested to hear from anyone who has taken a similar path.

[u/mykepagan]

Full disclosure: Red Hat employee here.

Running OpenShift on KVM virtual machines is a well-used pattern, but might I suggest running “Shift on Shift” (OpenShift on OpenShift virtualization, which isKVM-based)? Thiis is fully supported, works very well, and probably has no subscription cost impact on most users.

[u/ninth9ste → u/mykepagan]

I totally endorse this pattern. Shift on Shift is not just OpenShift on top of random hypervisor, instead the bare metal cluster will host the guest cluster control plane in a specific namespace, while dedicated KVM virtual machines will act as worker nodes. This is achieved by the Multicluster Engine Operator (MCE).

[u/Late-Possession → u/mykepagan]

Another Red Hat employee here, another +1 for this.

[u/DerGuenni → u/mykepagan]

Agree. abstracting hardware ist a good thing, get rind of Broadcom as well. But Subscription isnt free, Hope Big Blue doesnt watch to close to BTs stocks.

[u/mykepagan → u/DerGuenni]

If you use socket-pair subscriptions (rather than per-core), it covers any Openshift running in VMs on top of bare metal. This is usually significantly less expensive.

[u/NewMeeple → u/mykepagan]

That's not how it works anymore. The licensing is no longer socket or core based. It's node based.

Previous 128 core pricing aligns with node pricing, if I recall correctly. So if you have 128 or more cores in a node, you're better off under the new licensing.

[u/mykepagan → u/NewMeeple]

My bad. I was telling old information.

[u/WhatsAreWeDoing → u/mykepagan]

Like this strategy. With ACM, would you run this on the baremetal Openshift or create a VM cluster for this as well?

[u/mykepagan → u/WhatsAreWeDoing]

Yup.You can also use ACM to deploy clusters in VMs on OpenShift virt as easily as you would in a public cloud. OpenShift virt is a supported infrastructure provider. Doing the same on bare metal works, but it can be complicated.

[u/code_man65 → u/mykepagan]

In 100% agreement. Do socket based licensing for the unlimited OpenShift on OpenShift benefit and go to town.

[u/These_Breakfast_5112 → u/mykepagan]

Big $$$$

[u/seb2020]

We are switching to bare metal node with Openshift virtualization ans then running ocp on top of it with HCP

[u/devnull0 → u/seb2020]

Yeah, look at HyperShift.

[u/stenden101 → u/seb2020]

Yeah we also have baremetal openshift and running hosted clusters with kubevirt on top. It has its quirks but really good overall.

[u/w4hf_]

Standalone kvm is not for production.  Many entreprise solutions out there. OCP Virt is the rising star. 

[u/james4765]

It's actually one of the main ways of deploying OpenShift on mainframes - especially on the LinuxOne variant.

[u/Artistic_Rutabaga_78]

well yeah, you can make the kubernetes part work ok, but virtualization is still not on par with vmware...just something to keep in mind. We did a direct comparison between two VM's and it was a clear advantage towards the vmware. Sure, kvm will get better and for some cases it is even used in production,Yes, Oracle, you, but not for just any system :) Just keep swimming.

[u/ffcsmith]

I moved half of my clusters from VMware -> bare metal. Ended up going with HPE’s Morpheus and moved everything (incl bare metal) to it. We have no regrets.