"RMSProp" was the state-of-the-art form of #stochastic-gradient-descent #gradient-descent from 02012 to 02014, and was tossed out casually in a Coursera lecture by Geoffrey #Hinton.
on 02025-08-02#paper on escaping saddle points efficiently in #gradient-descent #optimization (with nearly-dimension-free performance)
on 02017-03-30“An overview of #gradient-descent #optimization #algorithms” “This blog post aims at providing you with intuitions towards the behaviour of different algorithms for optimizing gradient descent that will help you put them to use”
on 02016-05-04Quick #pdf #introduction to #gradient-descent #optimization #algorithms.
on 02016-01-13a #constraint solver and #optimizer for systems of #nonlinear equations using #Haskell to implement #gradient-descent, using #automatic-differentiation, compiled to #JS using Haste and drawn with #d3, equations rendered with MathJax. Some of the visualizations don’t work for me. Most of the examples are #kinematics. By the author of #Antimony!
on 02015-10-25