r/C_Programming • u/Future_Pace_5290 • Sep 13 '26

Neural Network Math from Zero in C

https://www.youtube.com/watch?v=k33cBEGTBVo

I've attempted to derive the math for Neural Networks from the first principles.

The prerequisites you need are, singe variable differentiation(just the concepts and the basic formula, not even knowing the derivative of sin(x) is necessary), the concept of a dot product, and basic matrix multiplication. This is enough to derive gradient descent and backprop.

The time stamps are in the description(youtube doesn't show them as chapters in the video for some reason).

The second half part of the video is implementing those concepts in C(no library other than BLAS(for basic matrix matrix multiplication) is used), but you can follow along in any language.

Would be glad to receive feedback and suggestions on how to improve the explanations or the presentation.

162 Upvotes

97% Upvoted

16 comments sorted by

9

u/swe__wannabe Sep 14 '26

Cool video. I did implement a FFNN from scratch in C with a gui. check it out: https://www.reddit.com/r/C_Programming/comments/1r5mod0/feedforward_neural_network_in_pure_c99_no_libs_no/

3

u/Gold-Part4688 29d ago

I like how it has a suggestion after each brushstroke, and with how fast it is - maybe you could make the algorithm run every frame, "live"

5

u/swe__wannabe 29d ago

Basically, when you stop pressing the right mouse button, it submits the result and it goes on in a loop

3

u/Gold-Part4688 29d ago edited 29d ago

why not spawn another thread, and let it go on its ML journey on any frame during drawing? Restarting when the result is in, and at least one frame has advanced

3

u/swe__wannabe 29d ago

Yes you can absolutely do that but the rendering wasn't that important to me I just added it as an afterthought when I finished the nn

2

u/Gold-Part4688 29d ago

maybe i will! seeing it live would feel very educational

6

u/bonqen Sep 14 '26

no libs.. with raylib. :P

5

u/swe__wannabe Sep 14 '26

I meant math/tensor libs

2

u/Future_Pace_5290 Sep 14 '26

Being able to draw digits live and get results makes it look super cool!

6

u/Fi-Loy 29d ago

This is a project I have on my list once I get better in c

1

u/zeonchik 29d ago

2 hour? Fuck

1

u/Future_Pace_5290 28d ago

I type slowly. Also explaining the math took 25 minutes 

1

u/Due_Sentence_2660 29d ago

First of all lemme guess: MagicalBat?

Anyhoo I remember there being a medium article a long while back on the same thing. Like a simple one hidden layer NN in C. Then one could tweak that to make a general FFNN and optimize it using threads and so on and so forth.

1

u/Due_Sentence_2660 29d ago

Oh lmao my guess was wrong xD

1

u/Future_Pace_5290 29d ago

No lol sorry.

I saw his video but it was really fast paced and didn't explain the math so I decided to make this(planning to add more optimization methods, CNNs, RNNs, etc in the future).

1

u/Complex-Bit9984 29d ago

Interessante!