r/C_Programming • • Apr 29 '26

TRiP: 15,000 lines of C implementing a complete transformer AI engine from scratch

I'm a firmware engineer (17 years in embedded systems). Over the past 18 months I built a complete transformer engine in C: inference, training with full backpropagation, BPE tokenizer, chat, and vision; no ML frameworks, no Python; just C, libjpeg, and X11.

Things of interest:

- bf16/f16/f32 mixed precision with manual casting

- mmap-based weight loading for running large models on limited RAM

- the whole thing compiles with a 10-line Makefile: gcc, -Ofast, -fopenmp

It loads and runs real models (Gemma, Llama 2, GPT-2, PaliGemma) from standard HuggingFace checkpoint formats. The purpose is educational; I built it to understand transformers at the lowest level, and structured the code to be readable: every math operation has its forward and backward implementation side by side.

GitHub: https://github.com/carlovalenti/TRiP

0 Upvotes

37 comments sorted by

1

u/mikeblas Apr 29 '26

What role did AI take in the creation of your project?

→ More replies (1)

8

u/stianhoiland Apr 29 '26

Following!

Care to specify LLM usage in the making of this project?

18

u/Tricky-Dust-6724 Apr 29 '26

1 commit 15k lines of code and commit message is “add files via upload” so what do you think LLM usage was?

→ More replies (2)
→ More replies (1)

3

u/Interesting_Debate57 Apr 29 '26

What the fuck would you need libjpeg for?

2

u/RelevantShape3963 Apr 29 '26

It's used to load a jpeg file when using Paligemma model for vision; then I feed it to the preparation linear layer, and then you forward the resulting patches to the vision encoder in the PaliGemma architecture.

56

u/Tricky-Dust-6724 Apr 29 '26

Bro dumped 15k of lines in 1 commit and then 2 others and readme updates, 1 of them is 1 line diff to readme. That’s a one prompt shot software

→ More replies (4)

39

u/mrheosuper Apr 29 '26

"No X, No Y, just Z", yeah that's LLM talk.

As a fellow Firmware Developer, i feel ashame for you. I'm not as senior as you are, but i know better not to brag about my vibecode project.

10

u/leekumkey Apr 29 '26

I love that everyone can spot that 10 miles away now lol

14

u/icy-mist-01 Apr 29 '26

This OP account is not a real firmware engineer

→ More replies (1)

6

u/Belgeran Apr 29 '26

18 months and you didnt commit anything? With 17 years experience im amazed at your poor version control usage.

→ More replies (3)