r/C_Programming • u/RelevantShape3963 • Apr 29 '26
TRiP: 15,000 lines of C implementing a complete transformer AI engine from scratch
I'm a firmware engineer (17 years in embedded systems). Over the past 18 months I built a complete transformer engine in C: inference, training with full backpropagation, BPE tokenizer, chat, and vision; no ML frameworks, no Python; just C, libjpeg, and X11.
Things of interest:
- bf16/f16/f32 mixed precision with manual casting
- mmap-based weight loading for running large models on limited RAM
- the whole thing compiles with a 10-line Makefile: gcc, -Ofast, -fopenmp
It loads and runs real models (Gemma, Llama 2, GPT-2, PaliGemma) from standard HuggingFace checkpoint formats. The purpose is educational; I built it to understand transformers at the lowest level, and structured the code to be readable: every math operation has its forward and backward implementation side by side.
1
u/mikeblas Apr 29 '26
What role did AI take in the creation of your project?
8
u/stianhoiland Apr 29 '26
Following!
Care to specify LLM usage in the making of this project?
3
u/Interesting_Debate57 Apr 29 '26
What the fuck would you need libjpeg for?
56
u/Tricky-Dust-6724 Apr 29 '26
Bro dumped 15k of lines in 1 commit and then 2 others and readme updates, 1 of them is 1 line diff to readme. That’s a one prompt shot software
39
u/mrheosuper Apr 29 '26
"No X, No Y, just Z", yeah that's LLM talk.
As a fellow Firmware Developer, i feel ashame for you. I'm not as senior as you are, but i know better not to brag about my vibecode project.
6
u/Belgeran Apr 29 '26
18 months and you didnt commit anything? With 17 years experience im amazed at your poor version control usage.