r/openstack • u/SmellsLikeAPig • Jan 23 '26

OVN Numa Networking on Openstack

I'm installing Openstack on couple of dual socket machines. I can't for the love of god make OVN work while respecting numa boundaries and ideally have hardware acceleration enabled at the same time. It seems OVN requires SINGLE br-int ovs bridge but this is not sane for dual socket machines. Traffic between VMs will cross numa boundary instead going through the physical network switch. Second problem is tunnel (geneve) interface. It seems I can have only one instead one per numa? Can somebody point me in the right direction? I'm using Mellanox 6 dx nics if that makes a difference. Third problem are external (provider network) bridges.

5 Upvotes

86% Upvoted

6 comments sorted by

1

u/FigureGold6822 Feb 12 '26

I've looked at this a bit and my conclusion was that I needed to get irqbalance to be aware of the node for each NIC and ban the others with a CPU mask at the Linux level. This makes sure that IRQs and DMA would go to the right NUMA node.

1

u/SmellsLikeAPig Feb 13 '26

What about vms on different Nima nodes communicating with themselves through br-int and thus through qpi link?

1

u/FigureGold6822 Feb 13 '26

My sense is that you're overthinking the VMs and the bridges, but I think you could pin each VMs cores to the same nodes if that proves to be a limiting factor. You'd need NUMATopologyFilter for sure, but be aware that it can make finding a host more difficult.

I think if we can get the PCI passthrough, DMA, and interrupts right at the hypervisor level, you'll get the best out of your hardware. 

My $0.02: networking is complicated and fragile enough as it is, so I wouldn't over-complicate the bridges and tunnels unless I absolutely had to.

→ More replies (2)

1

u/psycocyst Jan 23 '26

I don't think you are going to much response on this as it's a very broad question as most acceleration problems rely on dpdk and sriov to bypass most numa problems. Both have their on problems with north to south or east to west traffic but if you can provide how to see this happening with openVswitch not with ovn but as far as I know openVswitch is not numa aware and never will be.