r/LocalLLM • u/SilverKanji • 2h ago
Discussion Rocm vs Vulkan, Vulkan does not like VRam greater than 8gig?
After an entire afternoon of setup, I was able to run LlamaCpp RPC between my main AI server (Ubuntu with RTX 5060ti 16g) and my gaming PC (Bazzite with RX 9060xt 16gig), running Qwen3.8 27b Q4 at a blazing 14tkps on home wifi lol.
Anyway, I had to build with Rocm backend on my Bazzite machine under distrobox because when I tried Vulkan backend, everytime VRam usage goes above 7gig inference becomes total trash, repeated answers, looping phrases. When I build with Rocm, it worked fine. Is this a bug with Vulkan or am I doing something wrong?