Yann LeCun | May 18, 2021 | The Energy-Based Learning Model
Ғылым және технология
Title: The Energy-Based Learning Model
Speaker: Yann LeCun
Abstract: One of the hottest sub-topics of machine learning in recent times has been Self-Supervised Learning (SSL). In SSL, a learning machine captures the dependencies between input variables, some of which may be observed, denoted X, and others not always observed, denoted Y. SSL pre-training has revolutionized natural language processing and is making very fast progress in speech and image recognition. SSL may enable machines to learn predictive models of the world through observation, and to learn representations of the perceptual world, thereby reducing the number of labeled samples or rewarded trials to learn a downstream task. In the Energy-Based Model framework (EBM), both X and Y are inputs, and the model outputs a scalar energy that measures the degree of incompatibility between X and Y. EBMs are implicit functions that can represent complex and multimodal dependencies between X and Y. EBM architectures belong to two main families: joint embedding architectures and latent-variable generative architectures. There are two main families of methods to train EBMs: contrastive methods, and volume regularization methods. Much of the underlying mathematics of EBM is borrowed from statistical physics, including concepts of partition function, free energy, and variational approximations thereof.
Пікірлер: 8
At 8:17, the indices on the second and third lines look incorrect, shouldn't they be g_k, z_k, w_k instead of g_{k-1}, z_{k-1}, w_{k-1} in g?
@ramuk1127
2 жыл бұрын
No, look at the first line. Since z_k+1 is equivalent to g_k(z_k, w_k) it has to follow that the jacobian in the 2nd and 3rd lines can be written to compensate for backpropagation
@maxwellsdaemon7
2 жыл бұрын
@@ramuk1127 take a look at the previous slide and take the gradient with respect to z_k.
For more videos from the Mathematical Picture Language Tuesday seminar, see kzread.info/dron/rlS3CuPlahBp_M46fDaWVw.htmlvideos
Tiny insert brains, like a jumping spider has a 3d model of the world.
Energy VS Agency VS Time. If Agency could only be quantified…