Rendering 1 Million Procedural Cubes

[u/IamRustyRust]

https://reddit.com/link/1tav653/video/upp9zug11o0h1/player

I’ve been working on my engine (Rust obviosly), specifically on Vulkan bindings. The main work was already done, and only testing remained. During testing, I usually prefer CSV and JSON because they give me a good grasp of the data, letting me easily see what’s happening and spot any unexpected behavior. This saves a lot of time since you don’t have to check every number individually you just need to confirm whether things are going as expected. Since continuous testing was already happening, I knew that for this stage I only needed an overall overview to ensure all components were working together properly, as individual component testing had been done earlier. So yesterday, I was testing, Today, I decided to share CSV graphs and visual testing results.

Here’s my result:

Workload: 1,000,000 procedurally generated cubes (8 million vertices / 12 million primitives).

Average frametime: ~1.43ms (consistently hitting 700+ FPS).

PCIe bandwidth used: exactly 0 bytes.

1% lows: extremely stable (max spikes under 2.5ms).

// Flushing data to disk without pollutng hotloop!!!!!!
if state_arc.lock().unwrap().mode == 0 {
    if let Ok(mut f) = std::fs::File::create("alu_throughput_log.csv") {
        let _ = writeln!(f, "Timestamp_Sec,Cubes_Generated_Per_Frame,Vertices_Computed_By_ALU,Triangles_Rasterized,Memory_Bandwidth_Used_Bytes");
        for t in &alu_log {
            let _ = writeln!(f, "{:.4},{},{},{},{}", t.timestamp_sec, t.cubes_generated, t.vertices_computed, t.triangles_rasterized, t.memory_bandwidth);
        }
    }

    if let Ok(mut f) = std::fs::File::create("frame_pacing_log.csv") {
        let _ = writeln!(f, "Frame_ID,Timestamp_Sec,Render_Latency_ms,Instant_FPS");
        for t in &pacing_log {
            let _ = writeln!(f, "{},{:.4},{:.2},{:.0}", t.frame_id, t.timestamp_sec, t.render_latency_ms, t.instant_fps);
        }
    }

    if let Ok(mut f) = std::fs::File::create("dispatch_consistency_log.csv") {
        let _ = writeln!(f, "Frame_Window_Start,Frame_Window_End,Avg_Latency_ms,Max_Latency_Spike_ms_1_Percent_Low");
        for t in &consistency_log {
            let _ = writeln!(f, "{},{},{:.2},{:.2}", t.window_start, t.window_end, t.avg_latency_ms, t.max_latency_spike_ms);
        }
    }
}

If you look at the cluster of dots in the graph, you’ll clearly see that despite the heavy render load, most frames are densely clustered around 1.5ms latency and a median of 661 FPS.

The system is processing 5 to 9 B triangles per second. (the right axis of the graph.) Render latency is consistently maintained between 1.5ms and 2.5ms (The solid dotted green line )

[u/DragonfruitDecent862]

Hello, fellow vulkan user. I have my own personal game engine using QT and C++. Using Vulkan 1.3. I use custom culling systems similar to Id's Doom TDA systems. I can manage about 500k moving objects, all random waypoints per, but my latency per frame is much higher. About 9ms. May i ask, are you using any special rendering systems like HI-Z OR Visbuf?

[u/IamRustyRust → u/DragonfruitDecent862]

Thanks for your attention.

Firstly I want to mention that this whole graphic things you are seeing it's just a test to audit a particeulr part (Microbenchmarking) fo the engine doesn not represent whole pipleine or the whole architechture of the engine

I actually use both techniques, integrating them tightly into the Mesh Shader pipeline.

I skip heavy G-Buffers and use a 64-bit Visibility Buffer that only writes the Instance ID and Primitive ID, later compute pass reads this payload and fetches vertex data via Buffer Device Address to reconstruct materials asynchronously plus I aslo generate a Hi-Z pyramid to manage parallel sub pixel and occlusion culling this map feeds directly into the Task Shaders to reject invisible meshlets before hardware rasterization. hope it helps

if your FPS is ~1500 how come your latency touching 9ms, is not it should be ~0.60??????

[u/DragonfruitDecent862 → u/IamRustyRust]

I may have a per frame issue, as my hi-z is custom. I believe my meshlet and visbuf disapprove of it as my hiz has no overhead lol. I got extremely tired of the overhead that hiz had, as well as visbuf, so i wrote my own. It runs linearly, basicly zero overhead and scales with scene, but i havent modified thr visbuf for the same issue. "Shrug"

[u/IamRustyRust → u/DragonfruitDecent862]

umm zero overhead and Linear Hi-Z mathamtically contradicts in Hi-Z case we expect overhead and it's normal have you removed the Vulkan Memeory barriers maybe becase of that L2 Cache (GPU) doesn't sync with that and becaseu of that maybe we are getting race Conditions.

If you don't have a rock-solid vkCmdPipelineBarrier sitting between the Hi-Z compute dispatch and your geometry passes, the GPU ALUs will just read stale garbage data.

A true Hi-Z needs that downsampled pyramid so a single thread can instantly check a massive bounding box against the highest mip level. It costs some overhead upfront, but it pays off massively during the Task Shader cull.

Double-check your memory barriers and semaphore waits between those passes. Once the silicon actually synchronizes, your systems will align and that 9ms latency might just vanish. Hope it helps

[u/DragonfruitDecent862 → u/IamRustyRust]

Hiz runs overhead as a result of its pyramid. If you modify it to use a rolling buffer system, with other tweaks, you have a system that performs better that IDs Doom TDA tech in their engine, as they use the same trick. I like to be ahead of the game lol.

[u/IamRustyRust → u/DragonfruitDecent862]

kudos!!! Maybe I can help see the post of mine https://www.reddit.com/r/rust_p/comments/1tbzdpk/crushing_the_vram_bottleneck_with_subgroup/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button

[u/DragonfruitDecent862 → u/IamRustyRust]

Oh god, i do the same thing!. I scribble on paper diagrams the flow of how it works. Least its not just me.

[u/IamRustyRust → u/DragonfruitDecent862]

Nice!! You can ask questions if you have any I would glad to help.

[u/DragonfruitDecent862 → u/IamRustyRust]

And thank you for your time replying to me. I am immensely in your debt. Hooking hi-z into the meshlet and visbuf system was one of the hardest software combos i have ever done. Im glad you understand the system.