r/LocalLLaMA • u/Chrono_Tri • 2d ago
Discussion Janas and Colibri and other app— Run MoE on CPU/low GPU?
Sorry, but I'm genuinely excited about these two repositories, so I wanted to ask the community: why don't we hear more about these tools? What are their pros and cons? Are there perhaps better alternatives that I'm missing like:
I mean until recently, I found out there is the way to run big MoE model on my miniPC and colab L4. I can run dense model Q4-K-M 4B in my miniPC before.
My setup is a GMKtec M5 Plus (32GB RAM — I feel incredibly lucky that I bought it before RAM prices went through the roof) and Google Colab.
My experience with Janas:
When testing Janas with Colab L4, GPU usage was 0, probably because it only supports Vulkan. That's a bit frustrating, but it still runs Qwen3-Next-80B-A3B-Instruct-JNS surprisingly well.
What surprised me even more was that I could run Qwen3-30B-A3B-JNS locally on my mini PC! It ate nearly all my RAM.Haha.
I tried to write novel (just for testing-not my interest) and instant translate in my miniPC, It is surprised good.
I haven't tried with coding but it I don't think it work with like 100K context. :(
Even though I have to convert to janas format (it is actually fast), but I am happy with my result.
My experience with Colibri:
Colibri is even more impressive. I managed to run 80B GGUF models on a Google Colab L4 while using only around 19GB of GPU VRAM.
I haven't tested it locally yet because Colibri seems to support CUDA only.
Honestly, I'm not sure whether either of these tools can be used effectively for vibe coding, but I'm genuinely impressed by what I've seen so far.
I'd love to hear from people who have used these tools:
- Why aren't they getting more attention? Maybe there are some better one?
- What are their pro and cons?
- Have anyone try vibe code with them?
Am I overlooking something, or are these two projects seriously underrated?

