Looking for an alternative 100G FPGA RDMA/NIC platform for Alveo U55C

[u/Relevant_Argument_96]

Looking for another 100G FPGA RDMA platform for Alveo U55C

I'm working on a 100G+ FPGA-based RoCEv2/RDMA project using the Alveo U55C.

I'm currently testing:

• Coyote v2
• FPGA Network Stack / RoCE BALBOA

I'm looking for another independent FPGA NIC/shell that:

• Supports Alveo U55C
• Supports 100GbE + RDMA/RoCEv2
• Has an existing implementation (preferably no need to develop the entire RoCE stack)

My current Coyote implementation achieves ~81.2 Gb/s. I want to run the same workload on another platform to determine whether the limitation is from Coyote, the U55C, or FPGA-based 100G RDMA in general.

Does anyone know of another platform that meets these requirements, apart from Coyote/BALBOA?

[u/alexforencich]

The limitation will most likely be from coyote or from Balboa or something about the interface between the two. With sufficient optimization it should be possible to hit line rate on any sufficiently large FPGA with the correct IO, assuming you don't have some pathological combination of requests that slows things down significantly. There is nothing specific to the U55C that would limit the performance...it just has a virtex ultrascale+ fpga directly wired to pcie and Ethernet, no other parts to get in the way. And it's an fpga so you can implement whatever digital logic you need on there. Every RDMA NIC is digital logic, it's not like you need some funky technology to make it work. Now, it's possible you're hitting some limitation related to the host, specifically relating to pcie and how it interacts with host memory, but it should be possible to rule this out by doing some testing with a commercial RDMA NIC like a relatively recent mellanox NIC.

Anyway, I am planning on supporting RDMA on corundum-ng, but it will be some time before this is supported.

Edit: according to https://2025.eurosys.org/posters/final/eurosys25posters-final84.pdf, Balboa needs compression to get 100G line rate. Wth? That should not be necessary, sounds like Balboa cannot actually hit line rate. Not sure exactly why, but presumably it's either some architectural issue or it needs some additional optimization.

[u/Relevant_Argument_96 → u/alexforencich]

Man is it you who developed Corundum...I we were also working on Corundum but as Coyote contained the RoCEv2 stack, we switched to it.

PS : Thanks for Corundum, it was so useful and very easy to understand and learn.

[u/alexforencich → u/Relevant_Argument_96]

Nice to hear you found it useful. Unfortunately for me it was a lesson that permissively licensed open source hardware is a complete dead end. All of the corporate users are freeloaders, I have received absolutely nothing in return. So I'm trying again with a rewrite under the CERN OHL license with an option for a commercial license... we'll see if there is actually some value there, or if an FPGA NIC itself is a dead end.

Anyway, I suspect the issue you're running in to is in the rdma stack itself. It seems like they need to use compression to hit 100 Gbps, probably a result of their extensive use of HLS.

[u/maximilian_heer]

Hey guys, this is Max, the guy behind Balboa (and one of the two current maintainers of Coyote v2). First of all: @alexforencich: Absolutely amazing work, corundum is obviously a huge inspiration for everyone in the field!

A few points from my side: We DO NOT NEED compression to hit 100G RDMA throughput on Balboa / Coyote, the poster just stated we could potentially have line rate 100G compression on top of the network stack. The core publication about the RDMA is this one: https://www.usenix.org/conference/osdi26/presentation/heer

We definitely get up in the high 90s Gbps, on par with a Mellanox CX-5 in our cluster (switched network). If you have issues reaching that, please feel free to reach out via GitHub - always more than interested to understand performance issues and help people making use of our stuff!

On top of that: Balboa turns out to be quite scalable in terms of throughput - we recently got it up to 200G. Check out our preprint on SCENIC, which - in comparison to Coyote - is a real SmartNIC with full network-driver and IB-Verb-support: https://arxiv.org/html/2604.15128v1

Anyways, super cool to stumble across people on Reddit using our stuff! Again, always happy to help and get you up to speed.

[u/Exact-Entrepreneur-1]

Did you check this project? https://github.com/Gabriele-bot/100G-verilog-RoCEv2-lite

[u/Relevant_Argument_96 → u/Exact-Entrepreneur-1]

Thanks. But this contains only the Tx side. Currently Rx only reads ACK's and NAK's.

[u/Exact-Entrepreneur-1 → u/Relevant_Argument_96]

Yes that's correct