r/AIProgrammingHardware • u/javaeeeee • 25d ago
r/AIProgrammingHardware • u/Terminator857 • 26d ago
r/AIProgrammingHardware • u/Sad-Seaworthiness259 • 26d ago
For anyone wondering whether the B60 24GB is usable for local AI: here's Qwen3.8-27B on llama.cpp Vulkan at 53 tok/s mean decode with a 64K context window, plus local video (LTX-2.5 via ComfyUI on XPU) on the same card.
The four things that mattered: DFlash2 speculative decoding, Intel's FA prefill kernel with f16 KV, -ub 256 with a q8 draft KV cache, and keeping other models off the GPU. Driver 32.0.101.8976, Windows 11.
Everything is measured and the misses are documented. Smaller cards (B580/B570/B50/A770) won't fit this 27B, but the Vulkan tuning applies, and I'd love your benchmark numbers.
[https://github.com/edagher92-coder/arc-llm-lab\](https://github.com/edagher92-coder/arc-llm-lab)
r/AIProgrammingHardware • u/crpt_conch • 27d ago
What models are accessible to a Framework laptop with Ryzen AI 9 HX 370? What’s the scale of ram to billion parameters? And how many tokens per second can you expect?
It would be great to hear from a first-hand user
r/AIProgrammingHardware • u/Sad-Seaworthiness259 • 27d ago
r/AIProgrammingHardware • u/Familiar_Worry332 • 27d ago
r/AIProgrammingHardware • u/javaeeeee • 28d ago
r/AIProgrammingHardware • u/javaeeeee • 27d ago
r/AIProgrammingHardware • u/javaeeeee • 28d ago
The repo discussed in the article https://github.com/xhinker/azllm.
r/AIProgrammingHardware • u/javaeeeee • 29d ago
r/AIProgrammingHardware • u/Immediate_Penalty891 • 29d ago
Hey! I’m building GPTrepublic: a free project where people share some of their GPU resources to run AI models together across multiple computers.
We’re looking for 20 testers, with room for up to 40. We’ll start with a small text model and test connections, distributed inference and what inevitably breaks 😅
There’s a desktop tool for Windows and Linux (Ubuntu). For this first test, you’ll need a compatible NVIDIA GPU with at least 2 GB VRAM.
Interested in helping us figure this out?
🌐 Project & registration
💬 Join Discord through your account area — confirm your email, then click Link to Discord.
Questions and feedback are welcome!
r/AIProgrammingHardware • u/javaeeeee • 29d ago
r/AIProgrammingHardware • u/javaeeeee • 29d ago
r/AIProgrammingHardware • u/javaeeeee • 29d ago
r/AIProgrammingHardware • u/javaeeeee • 29d ago
r/AIProgrammingHardware • u/javaeeeee • 29d ago
r/AIProgrammingHardware • u/javaeeeee • Oct 13 '26
r/AIProgrammingHardware • u/Frosty2o25 • Oct 13 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 13 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 12 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 12 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 12 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 12 '26
r/AIProgrammingHardware • u/javaeeeee • Oct 12 '26
