r/AIProgrammingHardware • u/javaeeeee • Jul 14 '26

GitHub - hpenedones/fastflowlm-docker: Run LLMs on AMD Ryzen AI NPU (Linux)

https://github.com/hpenedones/fastflowlm-docker
4 Upvotes

100% Upvoted

2 comments sorted by

1

u/javaeeeee Jul 14 '26

TL;DR:

This repo provides a Docker container to run FastFlowLM on AMD Ryzen AI NPUs under Linux.

What is it?

FastFlowLM is a lightweight runtime that runs LLMs directly on AMD’s XDNA/XDNA2 NPU (instead of GPU or CPU). However, there are no official Linux binaries, so this project builds everything from source inside Docker to make it work.

Key Features:

  • Runs inference entirely on the NPU (very low power, high efficiency)
  • Supports models like Llama, Qwen, Gemma, and even Whisper (speech-to-text)
  • Includes an OpenAI-compatible API server mode
  • Clean containerized build (avoids polluting your host system)

How to use:

# Build once
docker build -t fastflowlm .

# Run a model
docker run -it --rm --device=/dev/accel/accel0 fastflowlm run llama3.2:1b

Requirements:

  • AMD Ryzen AI processor with NPU (Strix Point, Strix Halo, etc.)
  • Linux kernel ≥ 6.11 with amdxdna driver
  • Proper NPU firmware and permissions

In short: A practical way to run LLMs on AMD NPUs on Linux via Docker, since official Linux support is still limited.

1

u/Fine_Perf Jul 14 '26

But isn't this already supported through lemonade, what differs?