5.4 How Neural Networks Learn
They guess, measure the error, then use backpropagation and gradient descent to tweak every weight a tiny bit. Repeat a lot.
Chapter 5: Deep Learning and Neural Networks
Learning happens in a loop. Forward pass: data flows through the network and it makes a guess. Loss: you measure how wrong the guess was.
Backpropagation: the error is sent backwards through the network to work out how much each weight contributed to the mistake. Gradient descent: each weight gets nudged slightly in the direction that reduces the error.
Think of gradient descent like walking down a foggy hill: you can't see the bottom, but you can feel which way is downhill and take small steps. The size of each step is the learning rate - too big and you overshoot, too small and it takes forever.
Key takeaways
Forward pass → loss → backpropagation → update weights
Gradient descent = small steps downhill on the error
Learning rate = how big each step is