Building a Neural Network Framework from Scratch

I use PyTorch every day at work. It’s incredible - but I’ve always wondered: how does it actually work under the hood? How does backward() actually compute gradients through a neural network?

So I built my own minimal deep learning framework called Synap. It’s written in C++ for performance, with Python bindings via pybind11. No external ML libraries - just raw tensor operations and automatic differentiation from scratch.