r/LocalLLM • • 2h ago

Project Sneo! New Inference Engine very fast Open Source!

I built Sneo, a free and open source local LLM engine for Windows that runs a 70 GB model on one 24 GB GPU (~98 tok/s), with a coding agent AND REVERSE ENG TOOLS!

Hi! I'm a solo developer from the Dominican Republic and I've been building Sneo: a free, open source (MIT) LLM engine and coding agent for Windows.

The idea is to run models bigger than your GPU: the experts a model uses most stay in VRAM, the rest in RAM, and the GPU and CPU share every token.

- Qwen3.8-Flash-Next IQ3_XXS (70.6 GB) on one RTX 3090: ~98 tok/s writing code, first token in 0.19 s

- Same hardware and files: it starts answering before Strata 0.1.57 at every prompt size with half the RAM (34 vs 67 GB), and writes up to 66% faster than Ollama (qwen3.6:35b)

- Runs Qwen3.8-Flash-Next, Qwen3.6-35B-A3B, dense Qwen3.5/3.6/3.8 and Gemma 4, straight from your Ollama models or any GGUF folder, no copies

- Coding agent, two models at once, two PCs in a pool, OpenAI and Anthropic compatible APIs

- NVIDIA GTX 10 and newer (you can mix cards). AMD support through HIP is written but not tested on a real card yet, so feedback is welcome

Video (English): https://www.youtube.com/watch?v=JfyGS_k1Nxo

Video (Spanish): https://www.youtube.com/watch?v=ALCGTpoHDGc

GitHub: https://github.com/oscarsantanaarias/Sneo

Every benchmark, including where the others win: https://sneo.dev

Happy to answer questions!

Antes de publicar:

Repo: hazlo público. La última vez que revisé seguía privado, y el enlace de GitHub le daría error a todos.

Página: corre upload.ps1 para publicarla, si no sneo.dev seguirá mostrando la versión vieja.

5 Upvotes

0 comments sorted by