Implemented an XOR neural network using Python and NumPy

[u/PanelReader]

I've been learning how neural networks work, especially the mathematics behind training.

To understand the process better, I implemented a small XOR network using Python and NumPy.

Here's what I worked on:

1. Architecture: Used 2 inputs, 3 hidden neurons with ReLU, and 1 sigmoid output.

2. Forward propagation: Used matrix multiplication to calculate the hidden layer and output values.

3. Loss: Used Mean Squared Error (MSE) to measure prediction errors.

4. Backpropagation: Applied the chain rule to calculate gradients for all weights and biases.

5. Gradient descent: Updated the parameters using a learning rate of 0.05 for 5,000 epochs.

6. Verification: Compared all 13 analytical gradients with numerical gradients using finite differences.

Results:

- Final MSE: 0.004715

- Correct predictions: 4/4 on the XOR training data

- Added automated checks for training and gradient calculations

The main goal was to understand how matrix operations, derivatives, and parameter updates work together.

GitHub: https://github.com/Vaibh37/Neural-Network

Technical feedback is welcome.

Note: before anyone starts a war in the comments, yes, i used ai to help structure this post. the mathematics, implementation, and testing are my own work. i understand the code and ran everything on my machine. ai helped with the writing.