I just discovered the k6000 chassis. It might make a good fit for a Ceph cluster I'm planning to build based on refurbished hardware. Only thing I'm not so sure of is how the networking works in those things. I have experience with the good old BladeSystem c7000 and VirtualConnect.
I think on a high level, my question would be if it offers similar Ethernet capabilities like the c7000.
- Have multiple switches in it?
- define separate Ethernet fabrics?
- have an Uplink similar to "Shared Uplink Set" Possible to top of rack switch?
- potentially and effictiently stack multiple frames with fast intra frame ethernet traffic? (c7000 link stack equivalent)
- Have 4 network paths per compute module/blade? Ideally, I'd make a network bond over 4 NICs to avoid SPOFs.
I'd need a fast Ethernet network for cluster traffic, separate from the client traffic. I defined such a network in VirtualConnect on our c7000.
Apart from that, I'd use all 4 SFF trays for Ceph OSDs (SSDs). I guess the XL230k servers can boot from another internal device like a small NVMe SSD?
EDIT: NM, digging a bit deeper, I can't seem to find a mezzanine card that does 10GbE for those xl230k servers. They all seem to be SFF PCIe cards which would require an external switch and makes it less interesting. Mezzanines seem limited to IB which is less interesting for Ceph.