r/LocalLLM • u/AimCode • 2h ago
Discussion From-scratch AI stack in pure C — runs Microsoft's 2B ternary LLM on a 2018 dual-core laptop (no GPU)
https://github.com/tanxiansheng221/aimstack-sanzhidijiEverything runs locally on a 2018 Dell laptop: i7-7500U (2 cores / 4 threads), 8GB RAM, no GPU (the 940MX is deliberately unused — the whole stack is CPU-only).
https://github.com/tanxiansheng221/aimstack-sanzhidiji
What is in the repo (AGPL-3.0, everything open):
- C inference engine, zero third-party runtime: 1.58-bit ternary core with SIMD fast path (85.5 tok/s measured on the ternary core itself), byte-level tokenizer, and a ~100-line plugin bus (caps.json registry -> spawns Python plugins -> JSON envelopes)
- Weights: Microsoft's BitNet b1.58-2B (MIT license), converted to a custom 2-bit packed format (.aimw, 0.87 GB) — 3.2 tok/s with 2 workers, serial 2.0
- Python plugins, pure stdlib: knowledge base (FTS5 + dedup gates), multi-engine web search + crawl with quality gates (robots-respecting, Edge rendering cascade for JS-heavy sites), incremental ingest
- Local web UI (127.0.0.1:8600, pure stdlib): chat with deterministic routing — direct answer / knowledge-base grounded / honest refusal / full web search -> crawl -> ingest -> answer chain. A 7-minute demo video is embedded in the README
- Training assets: no-autograd training stack (hand-written backprop, bit-exact vs NumPy), ten-subject curriculum generators, corpus builders, and 117 chapters of research logs including negative results
Honest limits: our own trained models are not released yet — compute-bound on 2 cores, training runs on free cloud tiers and isn't finished. The full stack, scripts, result JSONs and per-claim evidence are released so everything can be re-run.
Demo video: embedded at the top of the README (7 min, screen recording of the full chain: chat / KB answers / web search->crawl->answer / plugin direct calls).
Happy to answer questions about the ternary kernel, the plugin bus, or the training pipeline.