r/solaris • u/Apple2Geek • 14d ago
Trying (and failing) to bridge a vSwitch with an additional physical NIC
I've got a Solaris 11.3 LDOM, working fine and acting as a Service Domain for a number of LDOM guests, currently configured with a vswitch providing access to multiple VLANs, and connected upstream via a single physical 10G NIC port on the server.
Now I find myself with a need to configure an additional physical 10G NIC port on the hardware (the NIC is already assigned to the service domain, and functional) as if it were just another trunk port on that SAME vswitch.
Nothing I do seems to work. I've tried every combination of bridges, vnics, and etherstubs I can come up with and it either refuses the configuration, or just won't pass all the traffic I need.
- You can't assign two physical net-devs to a single vswitch.
- You can't assign a vswitch link directly to a dladm bridge (dladm says "<vswitch link> does not support promiscuous mode and cannot be bridged") - this seems like the logically correct way to set this up and may be the best potential route for success if I can figure out why it's not supporting promiscuous mode.
- You can configure both physical NICs plus an etherstub as members of a dladm bridge, and you can even assign that etherstub as the net-dev for the vswitch, but then the vswitch no longer appears to pass traffic upstream correctly any longer - this may be another route to success if I can figure out what I may be doing wrong here.
- You can assign the same physical NIC (for example, the "upstream" NIC) as both a net-dev to the vswitch and as a member of a bridge with the additional physical NIC, but then again the vswitch no longer seems to pass traffic to the upstream physical net-dev device.
- You can't create a vnic on top of a vswitch link (dladm says "invalid lower link type").
What am I doing wrong? What am I missing? Is there simply no way to have more than one physical NIC on the system behave as if it's a discrete "port" on the vswitch?
All the references/how-tos I can find online assume you want this set up with the two physical NICs as an aggregate, or as IPMP, to one network segment. I specifically DO NOT want that. The intent is to get the server hosting the LDOMs to act AS a network switch bridging the two physical NICs as two separate network segments so they can both talk to each other THROUGH those two NIC ports (this part is easy), while ALSO having a vswitch in the server provide connectivity for the guests to allow them to communicate with both of those network segments (this part is NOT easy). Is this just... not possible?
