Deep Learning

Historical perspective on neural networks • Perceptron (Rosenblatt, 1962) : linear classifier • AdaLinE (Widrow, Hoff, 1962): linear regressor • Minsky/Papert (1969): first winter • Convolutional Neural Networks (1980, 1998) : great! • Multilayer Perceptron and backprop (Rumelhart, 1986) : great! • but it is hard to train, and the SVM come in the 1990s .... : second winter • 2006 : pretraining ! • 2012 : AlexNet on Imagenet (10% better on test than the 2nd) • Now on : lot of state of the art neural networks