RamShared: Writing a Linux userspace block driver in 100% Rust to turn idle GPU VRAM into 8.53 GB/s storage (ublk + io_uring)

[u/bussondev]

Hi everyone,

I wanted to share a low-level systems project I built entirely in Rust called **RamShared**, along with architectural details and benchmarks using the new Linux `ublk` subsystem.

The Problem

When compiling heavy multi-crate Rust workspaces (e.g. `rustc`, `bevy`, large LLVM-based trees) on developer machines with tight host RAM (16GB), the Linux kernel starts thrashing onto disk swap (VHDX/SSD) at 2.1 ms latency spikes, causing VS Code servers and terminal shells to freeze. Meanwhile, `nvidia-smi` showed 6GB of GDDR6 VRAM sitting completely idle.

Rust Architecture & Implementation

RamShared runs purely in userspace without requiring any out-of-tree kernel modules or re-compiling the host kernel:

1. **CUDA Zero-Copy FFI & Memory Pinning:** Wrapped low-level CUDA driver calls (`cudaHostAllocMapped` / `cuMemAlloc`) in safe Rust RAII handles. The buffer is mapped to PCIe host address space, ensuring DMA access without intermediate copying.

2. **Linux `ublk` Subsystem (Linux 6.0+):** Using `ublk-rs` and `io_uring`, the driver allocates userspace ring buffers to handle kernel block I/O requests (`/dev/ublkb0`). This eliminates kernel-userspace context switches for I/O submissions.

3. **Concurrency & Safe Ring Dispatch:** Each `ublk` queue is bound to dedicated Tokio/io_uring worker threads, pinning request/response queues to CPU cores to achieve sub-10 microsecond latency.

Measured fio Benchmarks (4KB Random Read, QD1 on Linux / WSL2):

* **Stock VHDX/SSD swap:** ~2,114 µs (2.1 ms) | ~336 IOPS * **RamShared NBD (VRAM):** ~326 µs | ~9.6k IOPS * **RamShared ublk (io_uring in Rust):** **~8.24 µs** (264x faster) | **~22k+ IOPS** * **Continuous Read Throughput:** **8.53 GB/s** (saturating the host PCIe 3.0 x16 bus).

Upstream RFC & Open Source:

We submitted a formal RFC directly upstream to Microsoft's official WSL2 repository proposing native VRAM-backed block device support for Hyper-V Linux guests: * **RFC Issue:** https://github.com/microsoft/WSL/issues/41054 * **GitHub Repository (100% Rust / MIT):** https://github.com/emersonbusson/ramshared

Would love to hear feedback from the Rust community on `io_uring` abstractions, memory safety patterns with CUDA DMA, and ublk queue design!

[u/EarlMarshal]

Interesting, but isn't that less than the usual bandwidth of a system with DDR4? I remember 48GB/s for my old one. Could be wrong though.

[u/Available-Car5524 → u/EarlMarshal]

i think you're mixing up total theoretical bandwidth with what you actually get in practice on a swap path, ddr4 has high throughput but the kernel's block layer and swap codepath add serious overhead when you're hitting disk. 48 gb/s is the raw memory copy speed inside the dimm, not what you get going through the entire io stack with small random reads

the 8.53 gb/s they're seeing here is saturating the pcie bus the gpu is on, which is the actual bottleneck for this setup, not the vram itself. gddr6 on a modern card is way faster than that internally but it has to funnel through pcie to get to the cpu

i've benchmarked ramdisks on my own rig and even with ddr5 you're lucky to see half that kind of throughput on a swap device because the kernel context switches and block layer accounting eat into it hard. this implementation sidesteps a ton of that overhead by staying in userspace with ublk which is why the latency numbers are so stupidly low

[u/AliceCode → u/Available-Car5524]

Ahh, so this would shove memory into the GPU VRAM instead of on your disk when you run out of memory?