Abstract

In 2006, there was a resurgence of interest in deep neural networks. This was triggered by the discovery that there was a simple and effective way to pre-train deep networks as generative models of unlabeled data. The pre-trained networks could then be fine-tuned discriminatively to give excellent performance on labeled data. In this lecture, I will describe the pre-training procedure used for Deep Belief Nets and show how it evolved from an earlier training procedure for Boltzmann machines that was theoretically elegant but too inefficient to be practical. I will also show how the pre-training procedure overcame a major practical problem in training densely connected belief nets.

Biography

Geoffrey Hinton received his PhD in Artificial Intelligence from Edinburgh in 1978.  After five years as a faculty member at Carnegie-Mellon he became a fellow of the Canadian Institute for Advanced Research and moved to the Department of Computer Science at the University of Toronto where he is now an Emeritus Distinguished Professor. He is also a Vice President & Engineering Fellow at Google and Chief Scientific Adviser of the Vector Institute.

Geoffrey Hinton was one of the researchers who introduced the backpropagation algorithm and the first to use backpropagation for learning word embeddings. His other contributions to neural network research include Boltzmann machines, distributed representations, time-delay neural nets, mixtures of experts, variational learning and deep learning.  His research group in Toronto made major breakthroughs in deep learning that revolutionized speech recognition and object classification.

Geoffrey Hinton is a fellow of the UK Royal Society, a foreign member of the US National Academy of Engineering and a foreign member of the American Academy of Arts and Sciences. His awards include the David E. Rumelhart prize, the IJCAI award for research excellence, the Killam prize for Engineering, the IEEE Frank Rosenblatt medal, the IEEE James Clerk Maxwell Gold medal, the NEC C&C award, the BBVA award, the Turing award, and the NSERC Herzberg Gold Medal which is Canada's top award in Science and Engineering.