Optimisers — SGD, RMSProp & Adam — A Complete Reference Guide
Imagine being blindfolded in a giant valley, trying to find the lowest point — feeling the slope, stepping downhill, again and again. That is exactly what an optimiser does, across millions of dimensions. The three that power 90% of deep learning: SGD with momentum, RMSProp, and Adam (200,000+ citations).