The Deep Learning Revolution and Its Implications for Computer Architecture and Chip Design
Jeff Dean, Google, Mountain View, CA
The past decade has seen a remarkable series of advances in machine learning, and in particular deep learning approaches based on artificial neural networks, to improve our abilities to build more accurate systems across a broad range of areas, including computer vision, speech recognition, language translation, and natural language understanding tasks. In this talk, I will highlight some of these advances, and their implications on the kinds of computational devices we need to build, especially in an era where general purpose computers are no longer improving their performance significantly year-over-year. I'll also discuss some of the ways that machine learning may also be able to help with some aspects of the circuit design process. Finally, I'll provide a sketch of at least one interesting direction towards much larger-scale multi-task models that are sparsely activated and employ much more dynamic, example- and task-based routing than the machine learning models of today.