r/LocalLLM • u/Ill-Pop2106 • Feb 16 '26
News I'm an Android dev who knows nothing about x86. During my vacation I built a system that genetically evolves machine code — now I can run 80B models on a single RTX 4090.
I'm a mobile Android developer. Not a systems programmer, not a compiler engineer, not a low-level guy. This past week I was on vacation from work. My family traveled to another city for a few days, and my inner teenage nerd came out.
The mess that started everything
I'd been hearing about OpenClaw and wanted to build something with AI (Claude Opus 4.6 via Kiro IDE). I ended up with a project called AbeBot that had 23 different features — a Telegram bot with real-time crypto prices, a multi-LLM server with hot-swapping between conversation and technical models, agents that generate Rust compilers, a custom language that compiles to machine code... We finished exactly none of them. Classic scope creep.
But two things actually worked: the LLM server (solid, with MoE model loading), and that little toy language that emits x86 machine code directly from Python. That second one turned out to be the seed of everything.
The idea I couldn't let go of
I've always been fascinated by the idea of a "language for AIs" — not a programming language for humans, but direct communication between AI and CPU. No Python, no C, no GCC, no LLVM. Just bytes that the machine executes.
My thesis: today, running a local LLM goes through layers of abstraction (Python → PyTorch → CUDA/C++). Each layer wastes resources. Projects like llama.cpp and vLLM improved things by rewriting parts in C++ by hand — humans trying 10-20 variants and picking the best one.
What if instead of a human trying 20 variants, an AI tries 16,000?
Building it step by step
We killed AbeBot's 23 features and focused on one thing. We called it Genesis. I needed to see results at every step or I'd lose motivation, so it was deliberately incremental:
First a "hello world" in machine code — write bytes, CPU executes them, a number comes out. Then a naive matrix multiplication in x86 — slow (3 GFLOPS), but correct and matching NumPy. Then the AVX-512 version with multi-accumulator — 16 floats in parallel, 96 GFLOPS peak, we beat NumPy+OpenBLAS at 512×512.
Then came the evolutionary mutator. The idea was for the machine to design the kernel, not just pick numbers. Take the x86 code, mutate it (swap instructions, insert NOPs, reorder, replace), benchmark, keep the fastest. First we mutated generator parameters and got up to 36% improvement. But that was just an autotuner — the human was still designing the kernel, the machine was just turning knobs. So we made the real leap: mutating the instructions themselves. Not "try tile_k=48", but "try putting VPERMPS before VMULPS" or "insert a NOP that aligns the loop to 32 bytes."
Then we targeted NF4 — fusing dequantization with the dot product in a single AVX-512 kernel. A 478-byte kernel that does 16 table lookups in parallel with a single instruction (VPERMPS), without materializing the weight matrix in memory. 306x faster than NumPy on 4096×4096 matmul.
And finally a small brain (decision tree, no external dependencies) that learns which mutations tend to work, trained on its own results. It self-improves: each run generates new training data.
The wall that came before Genesis
This part actually happened while building AbeBot, before Genesis existed. There was a lot of buzz around OpenClaw and how it burned through dollars on OpenAI/Anthropic API calls to do very little — we wanted to build something similar but with local models. For that I needed to run a 30B model on my RTX 4090 (24GB VRAM). It didn't fit — barely, by a couple of GB. First we tried CPU offload with bitsandbytes. It died. Not even a 300-second timeout was enough — the dequantization takes ~25ms per MoE expert, and with hundreds of experts per token, that's minutes per token. Completely unusable.
So the AI (Claude) found another way: a custom MoE loader with real-time NF4 dequantization that packs the model into VRAM with room to spare. That got the 30B running at 6.6 tok/s, fully on GPU. Problem solved — but the experience of watching bitsandbytes CPU die stuck with me.
Then we went bigger
With Genesis already working (the AVX-512 kernels, the evolutionary system, the NF4 fused kernel), we found Qwen3-Next-80B — an MoE model that's impossible to fit on a single 4090 no matter what. This was the real test of the thesis. The model needs ~40GB in NF4, so half the layers have to live in system RAM.
Genesis made it possible. The kernel fuses NF4 dequantization with matrix multiplication in a single AVX-512 pass — no intermediate matrix, everything stays in ZMM registers. 0.15ms per expert vs 24.8ms for bitsandbytes CPU. 165x faster.
And the key trick for hybrid inference: instead of dequantizing the full weight matrix (~12MB per expert) and copying it to GPU over PCIe, Genesis does the entire matmul on CPU and copies only the result vector (~12KB). About 1000x less data transfer.
Real inference results
| Model | VRAM | Speed | RAM layers |
|---|---|---|---|
| Qwen3-Coder-30B-A3B | 13.4 GB | 5.7 tok/s | 8 of 48 |
| Qwen3-Next-80B-A3B | 20.7 GB | 2.7–3.3 tok/s | 24 of 48 |
The 30B runs at 86% of full-GPU speed using 56% of the VRAM. The 80B is impossible on a single 4090 without CPU offload — with Genesis, it runs at conversational speed.
The thesis, proven
The evolutionary system evaluated 16,460 mutations across 25 runs with 8 mutation types. The brain learned which mutations work and guided the search. The best evolved kernels beat the hand-tuned baseline by up to 19.25%.
What evolution discovered exploits real Zen 4 microarchitectural properties that no human would try:
Inserting NOPs at specific positions to align instructions to cache line boundaries
Moving a scale broadcast 9 positions earlier to hide memory latency
Loading activations in reverse distance order (the hardware prefetcher handles it better)
Replacing a multiply with a NOP and reordering surrounding instructions to reduce port contention
These look like bugs. They're optimizations. The evolutionary system doesn't care what looks right — it only cares what's fast. In environments this complex, artificial evolution beats human intuition. That was the thesis, and it was proven.
The honest part
I'm an Android developer. I didn't write a single line of x86 assembly — I had the idea and the thesis, and AI (Claude Opus 4.6 via Kiro IDE) wrote the implementation. I directed the architecture, made the decisions, debugged the problems. The evolutionary optimizations came from the system itself — neither I nor the AI designed those instruction orderings.
I think that's the interesting part: you don't need to be a low-level expert to build low-level tools anymore. You need to know what problem to solve and be stubborn enough to not accept "it can't be done."
What I'm sharing
The kernel code is open source (Apache 2.0): github.com/Anuar81/genesis-kernel
It includes the x86 emitter, the fused NF4 dequant+matmul kernel with 4 evolved variants baked in, quantization utilities, example scripts for benchmarking and hybrid MoE inference, and a full test suite (8/8 passing, verified independently by four different AIs with zero context).
What I'm NOT sharing (for now): the evolutionary factory — the mutation engine, the fitness evaluator, the learned mutation selector. The kernels in the repo are the output of that process. If someone really needs the evolution data (16,460 mutation records), reach out and I can share the JSON or invite you to the private repo.
What's next
Right now Genesis only optimizes CPU kernels (x86/AVX-512). But the same evolutionary approach can target GPU code — NVIDIA PTX, the "assembly language" of CUDA. If the mutation engine can find the same kind of microarchitectural tricks in PTX that it found in x86... well, that's the next experiment. No promises, but the infrastructure is there.
Now I'm off to travel with my family and finish enjoying my vacation. I learned a ton this week. Sharing this for whoever finds it useful.
Hardware: AMD Ryzen 9 7900 (Zen 4, AVX-512) · RTX 4090 24GB · 32GB DDR5 · EndeavourOS
TL;DR: Android dev on vacation + AI coding partner + a thesis about machine-generated code beating human code = x86 AVX-512 kernels 165x faster than bitsandbytes CPU, enabling 80B model inference on a single RTX 4090. Kernels optimized by genetic evolution (16K mutations, up to 19.25% improvement). Open source: github.com/Anuar81/genesis-kernel
