In our third meetup we tried to take a closer look at optimization algorithms which are frequently used in the context of training of Artificial Neural Networks.
News
- Character Sequence Models for ColorfulWords (http://colorlab.us)
- Deep Learning ’ahem’ detector (https://github.com/worldofpiggy/deeplearning-ahem-detector)
- Judging a Book By its Cover (https://arxiv.org/abs/1610.09204)
- LipNet: Sentence-Level Lipreading (http://openreview.net/forum?id=BkjLkSqxg)
Presentation
The presentation used on this meetup, along with a list of references used in its preparation can be found here.
Note that the presentation was in many ways inspired (sometimes a bit too much) by this article.
More Pointers
- http://blog.mrtz.org/2013/09/07/the-zen-of-gradient-descent.html
- http://int8.io/comparison-of-optimization-techniques-stochastic-gradient-descent-momentum-adagrad-and-adadelta/
- http://www.benfrederickson.com/numerical-optimization/ (A very nice and fun introduction to numerical optimization in general)