This is a talk delivered at the (usually not recorded) weekly journal club "Deep Learning: Classics and Trends" (mlcollective.org/dlct/ ).
Speaker: Rishabh Agarwal
Title: Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress
Abstract: Learning tabula rasa, that is from scratch without using any learned knowledge, is the prevalent workflow in reinforcement learning (RL) research. However, RL systems, when applied to large-scale settings, rarely operate tabula rasa. Additionally, the inefficiency of deep RL typically excludes researchers without access to industrial-scale resources from tackling computationally-demanding problems. To address these issues, our NeurIPS 2022 paper presents an alternative way of doing RL research, which we call reincarnating RL, where prior computational work (e.g., learned policies) is reused or transferred between design iterations of an RL agent, or from one RL agent to another. Through a case study and experimental results, I’ll try to illustrate how reincarnating RL can improve real-world RL adoption and help democratize RL further. See agarwl.github.io/reincarnating_rl for more details.
Speaker's bio: Rishabh is a senior research scientist in the Google Brain Team in Montréal. He currently holds the record for giving the most number of talks at DLCT. Previously, he spent a year at Geoffrey Hinton's team in Google Brain, Toronto. His research work mainly revolves around deep reinforcement learning (RL), often with the goal of making RL methods suitable for real-world problems, and includes an outstanding paper award at NeurIPS 2021.
Paper link: agarwl.github.io/reincarnatin...