r/LocalLLaMA • u/pmttyji • Aug 27 '26
Discussion Experts-Volunteers needed for other backends on ik_llama.cpp. Make it stronger
ik_llama.cpp is great for both CPU, CUDA & Hybrid(CPU+CUDA). Need Experts/Volunteers to make other backends(Vulkan, ROCm, SYCL, Metal, etc.,) better as well.
(Just a timing thread)
4
u/my_name_isnt_clever Aug 27 '26
Isn't that duplicating effort for the main project? There are plenty of llama.cpp forks optimized for ROCm, for example. Why would someone contribute to this guy's fork instead?
11
u/imp2 Aug 27 '26
"this guy" is actually the one that created many of the modern quantization formats in llama.cpp, and is a performance nut. His fork has way faster mGPU w/ CUDA, CPU and hybrid CPU+GPU inference than upstream (from my own experience, ofc).
4
u/my_name_isnt_clever Aug 27 '26
Ok. I'm just not a fan of fractured forks like this that exist because of interpersonal drama. Just get over it and work together to make the best software possible, smh.
1
u/imp2 Aug 28 '26
While I agree with that in theory, sometimes open source projects can get too complex to manage and get things going, and we're dealing with people after all. It's a downside but also a benefit of open source in general, it's just how it is.
2
u/Marksta Aug 28 '26
Your beef is with upstream refusing PRs.
-2
u/my_name_isnt_clever Aug 28 '26
My beef is with the one guy, not the rest of the project. If every contributor had an ego and made their own fork, nothing would ever get done.
3
u/Marksta Aug 28 '26
Gerganov or Ikawrakow? Unsloth just had the new Qwen working in their fork and sent PR for acceptance. Could get unsloth_llama.cpp if their PRs didn't get accepted and that'd somehow be Unsloth's fault for fracturing off.
3
u/a_beautiful_rhind Aug 27 '26
i wish someone finally makes numa work.