This talk will discuss past, present, and future of predictive coding, with a special emphasis on potential applications in machine intelligence. To do this, we will first summarize the journey that has brought predictive coding from being considered a compression algorithm for time series data, to its contemporary role as an evidence-maximization framework for a specific type of generative models. Then, we will provide an informal definition of what predictive coding is today, and use it as an umbrella for different algorithms that fall under this definition. We will then contextualize where predictive coding stands in a community where the vast majority of the research is focused on the improving of backpropagation-based models. To do so, we will survey theoretical results that show differences and similarities among the two algorithms. A special focus is then placed on the strong points of predictive coding, such as its natural robustness, flexibility, and biological plausibility. Predictive coding indeed shows advantages on problems that are faced by biological organisms, such as continual learning, online learning, and learning from a small amount of data. Furthermore, the bio-plausibility of its update rules is particularly suited for implementations on neuromorphic chips. So, where is predictive coding headed? The conclusive part of this talk will be focused in providing an answer to this question, and will cover a list of the challenges that need to be met to allow prevent predictive coding to be used in practical tasks.
Talk by Dr Tommaso Salvatori, entitled "On the past, present, and future of predictive coding." Presented at the 4th International Symposium on the Mathematics of Neuroscience, Rhodes, 2023 (www.neuromonster.org). Recorded and hosted with generous funding from the Diverse Intelligences Summer Institute (www.disi.org) and Google DeepMind (deepmind.google/).
© Thinking About Thinking, Inc, a 501(c)3 Non-profit registered in New Jersey, USA.