Gradient Descent Simulator
See how optimisers find their way down a loss surface. All four start from the same point and race to the bottom on a contour map, so you can see why momentum helps in a ravine and what a learning rate that is too high does.
What you can do
- SGD, momentum, RMSProp and Adam side by side
- Six surfaces, including a ravine, a saddle and the Rosenbrock valley
- Tap the map to choose where they start
- Adjustable learning rate and gradient noise
- A live loss curve for each optimiser