Abstract

Humans are much better than current AI systems at generalizing out-of-distribution. What ingredients can bring us closer to that level of competence? We propose 4 ingredients to be combined: (a) meta-learning (to learn end-to-end to generalize to modified distributions, sampled from a distribution over distributions), (b) designing modular architectures with the property that modules are fairly independent of each other and interacting sparsely while made to be composed in new ways easily, (c) capturing causal structure decomposed into independent mechanisms so as to correctly infer the effect of interventions by agents which modify the data distribution, and (d) building better and more stable models of the invariant properties of possibly changing environments by taking advantage of the interactions between the learner and its environment to learn semantic high-level variables and their interactions, i.e., adopting an agent perspective on learning to benefit deep learning of abstract representations. The last ingredient implies that learning purely from text is not sufficient and we need to strive for learning agents which build a model of the world, to which linguistic labels can be associated, i.e., performing grounded language learning. Whereas this agent perspective is reminiscent of deep reinforcement learning, the focus is not on how deep learning can help reinforcement learning (as a function approximation black box) but rather how the agent perspective common in reinforcement learning can help deep learning discover better representations of knowledge.

Biography

Yoshua Bengio is recognized as one of the world's leading experts in artificial intelligence and a pioneer in deep learning. Following his studies in Montreal, culminating in a Ph.D. in computer science from McGill University in 1991, Professor Bengio did postdoctoral studies at the Massachusetts Institute of Technology (MIT) in Boston. Since 1993, Yoshua Bengio has been a professor in the Department of Computer Science and Operational Research at the Université de Montréal. In addition to holding the Canada Research Chair in Statistical Learning Algorithms, he is also the founder and scientific director of Mila, the Quebec Institute of Artificial Intelligence, the world's largest university-based research group in deep learning. He is also the Scientific Director of IVADO.

His contribution to research is undeniable. In 2018, Yoshua Bengio is the computer scientist who collected the largest number of new citations in the world, thanks to his three books and some 500 publications. His ultimate goal is to understand the principles that lead to intelligence through learning and his research has earned him multiple awards. In 2017, he was made an Officer of the Order of Canada and a Fellow of the Royal Society of Canada, in addition to receiving the Prix Marie-Victorin and being named Scientist of the Year by Radio-Canada. In 2018, he was also awarded the 50th Anniversary Medal by Quebec's Ministère des Relations internationales et de la Francophonie. In 2019, he was awarded the Killam Prize as well as the 2018 Turing Award, considered to be the Nobel prize for computing. These honours reflect the profound influence of his work on the evolution of our society.