Variational Autoencoders

Ғылым және технология

In this episode, we dive into Variational Autoencoders, a class of neural networks that can learn to compress data completely unsupervised!
VAE's are a very hot topic right now in unsupervised modelling of latent variables and provide a unique solution to the curse of dimensionality.
This video starts with a quick intro into normal autoencoders and then goes into VAE's and disentangled beta-VAE's.
I aslo touch upon related topics like learning causal, latent representations, image segmentation and the reparameterization trick!
Get ready for a pretty technical episode!
Paper references:
- Disentangled VAE's (DeepMind 2016): arxiv.org/abs/1606.05579
- Applying disentangled VAE's to RL: DARLA (DeepMind 2017): arxiv.org/abs/1707.08475
- Original VAE paper (2013): arxiv.org/abs/1312.6114
If you want to support this channel, here is my patreon link:
/ arxivinsights --- You are amazing!! ;)
If you have questions you would like to discuss with me personally, you can book a 1-on-1 video call through Pensight: pensight.com/x/xander-steenbr...

Пікірлер: 455

  • @abaybektursun
    @abaybektursun6 жыл бұрын

    Variational Autoencoders starts at 5:40

  • @pouyan74

    @pouyan74

    4 жыл бұрын

    You just saved five minutes of my life!

  • @moazalomary8123

    @moazalomary8123

    3 жыл бұрын

    @@pouyan74 no the first part was necessary...

  • @selmanemohamed5146

    @selmanemohamed5146

    3 жыл бұрын

    @@moazalomary8123 you think someone would enter a video about Variational Autoencoders if he doesn't know what Autoencoders are

  • @moazalomary8123

    @moazalomary8123

    3 жыл бұрын

    @@selmanemohamed5146 yeah i did... 😂😂😂 and i was lucky he explained both 😎🙌😅 + the difference between them and that's the important part

  • @moazalomary8123

    @moazalomary8123

    3 жыл бұрын

    @Otis Rohan Interested

  • @atticusmreynard
    @atticusmreynard6 жыл бұрын

    This kind of well-articulated explanation of research is a real service to the ML community. Thanks for sharing this.

  • @vindieu

    @vindieu

    Жыл бұрын

    Except for "Gaussian" that is weirdly russian pronunced "khaussian" wat?

  • @arkaung
    @arkaung6 жыл бұрын

    This guy does a real job of explaining things rather than hyping up things like "some other people".

  • @malharjajoo7393

    @malharjajoo7393

    4 жыл бұрын

    are you referring to Siraj Raval? lol

  • @mubangansofu7469

    @mubangansofu7469

    2 жыл бұрын

    @@malharjajoo7393 lol

  • @ambujmittal6824
    @ambujmittal68244 жыл бұрын

    Your way of simplifying things is truly amazing! We really need more people like you!

  • @obadajabassini3552
    @obadajabassini35526 жыл бұрын

    A really great talk! I have been reading about VAE a lot and this video helps me to understand it even better. Thanks!

  • @debajyotisg
    @debajyotisg5 жыл бұрын

    I love your channel. A perfect amount of technicality so as to not scare off beginners, and also keep the intermediates/ experts around. Brilliant.

  • @jingwangphysics
    @jingwangphysics2 жыл бұрын

    The beta-VAE seems enforcing a sparse representation. It magically picks the mostly relevant latent variables. I am glad that you mentioned ‘causal’, because that’s probably how our brain deals with high dimensional data. When resources are limited (corresponding to use large beta), the best representation turns out to be a causal model. Fascinating! Thanks

  • @antonalexandrov4159
    @antonalexandrov41592 жыл бұрын

    Just found your channel and I realize how with some passion and effort you explain things better than some of my professors. Of course, you don't go into too much detail but putting together the big picture comprehensively is valuable and not everyone can do it.

  • @paradoxicallyexcellent5138
    @paradoxicallyexcellent51385 жыл бұрын

    I was very interested in this topic, read the paper, watched some videos, read some blogs. This is by far the best explanation I've come across. You add a lot of value here to the original rapper's contribution. It could even be said you auto-encoded it for my consumption ;)

  • @adityakapoor3237
    @adityakapoor32372 жыл бұрын

    This guy was a VAE to the VAE explanation. Really need more of such explanations with the growing literature! Thanks!

  • @ujjalkrdutta7854
    @ujjalkrdutta78545 жыл бұрын

    Really liked it. Firstly giving an intuition of the concept, its application and then to the objective function while explaining its individual terms, in a way everyone can understand, it was simply professional and elegant. Nice work and thanks!

  • @515nathaniel
    @515nathaniel4 жыл бұрын

    "You cannot push gradients through a sampling node" TensorFlow: *HOLD MY BEER!*

  • @nabeelyoosuf
    @nabeelyoosuf4 жыл бұрын

    Your explanations are quite insightful and flawless. You are are a gifted explainer! Thanks for sharing them. Please keep sharing more.

  • @ashokkannan93
    @ashokkannan935 жыл бұрын

    I would like to see more videos from you. Clear explanation of concept and gentle presentation of math. Great job!

  • @hcgaron
    @hcgaron5 жыл бұрын

    I discovered your channel today and I'm hooked! Excellent work. Thank you so much for your hard work

  • @isaiasprestes
    @isaiasprestes6 жыл бұрын

    Great! No BS, strait and plain English! That`s what I want!! :) Congratulations!

  • @Zahlenteufel1
    @Zahlenteufel1 Жыл бұрын

    Bro this was insanely helpful! I'm writing my thesis and am missing a lot of the basics in a lot of relevant areas. Great summary!

  • @giorgiozannini5626
    @giorgiozannini56263 жыл бұрын

    wait how did I not know of this channel. Beautiful explanation, perfectly clear. Thanks for the awesome work!

  • @moozzzmann
    @moozzzmann5 ай бұрын

    Great Video!! I just watched 4 hours worth of lectures, in which nothing really became clear to me, and while watching this video everything clicked! Will definitely be checking out your other work

  • @ejkitchen
    @ejkitchen6 жыл бұрын

    Your videos are quite good. I am sure you will get an audience in no time if you continue. Thank you so much for making these videos. I like the style you use a lot and love the time format (not too short and long enough to do a good overview dive). Well done.

  • @ArxivInsights

    @ArxivInsights

    6 жыл бұрын

    Thank you very much for supporting me man! New video is in the making, I expect to upload it hopefully somewhere next week :)

  • @adityamalte476
    @adityamalte4766 жыл бұрын

    Really appreciate your effort of simplifying research papers for viewers.Keep it up.I want more such videos

  • @TheJysN
    @TheJysN3 жыл бұрын

    I hand such a hart time understanding the Reparameterization trick, now i finally got it. Thanks for the great explanation. Would love to see more Videos from you.

  • @dimitryversteele2410
    @dimitryversteele24106 жыл бұрын

    Great video ! Very clear and understandable explanaitions of hard to understand topics.

  • @aryanirvaan
    @aryanirvaan2 жыл бұрын

    Dude what a next level genius you are! You made them so easy to be understood, and just look at the quality of the content. Damn bro!🎀

  • @reinerwilhelms-tricarico344
    @reinerwilhelms-tricarico3444 жыл бұрын

    Great! Crisply clear explanations in such a short time.

  • @rylaczero3740
    @rylaczero37406 жыл бұрын

    Bloody nicely explained than the Stanford people. Subscribed to the channel, I remember watching your first video on Alpha, but didn't subscribed then, I hope there will be more content on channel with same level of quality, otherwise its hard for people to stick around when the reward is sparse.

  • @venkatbuvana
    @venkatbuvana5 жыл бұрын

    Thanks a lot for sharing such a succinct summarization of VAEs. Very helpful!

  • @DistortedV12
    @DistortedV124 жыл бұрын

    This was very lucid. You are gifted at explaining things!

  • @DanielWeikert
    @DanielWeikert6 жыл бұрын

    Great work. Thanks a lot! Highly appreciate your effort. Creating these videos takes time but I still hope you will continue.

  • @MonaJalal
    @MonaJalal4 жыл бұрын

    hands down this was the best autoencoder and variational autoencoder tutorial I found on Web.

  • @nohandlepleasethanks
    @nohandlepleasethanks6 жыл бұрын

    Great explanations. This filled two crucial gaps in my understanding of VAEs, and introduced me to beta-VAEs.

  • @lisbeth04
    @lisbeth045 жыл бұрын

    I love you. I spent so long on this and couldn't understand the intuition behind it, with this video I understood immediately. Thanks

  • @agatinogiulianomirabella6590
    @agatinogiulianomirabella65902 жыл бұрын

    Best explanation found on the internet so far. Congratulations!

  • @kalehermit
    @kalehermit4 жыл бұрын

    Thank you very much, this is the first time I understand the benefit of reparameterization trick.

  • @get.ai.enabled
    @get.ai.enabled6 жыл бұрын

    This is a LIT channel for watching alongside papers. Thanks

  • @davidm.johnston8994
    @davidm.johnston89946 жыл бұрын

    Great videos man, keep them going, you're gonna find an audience!

  • @achakraborti
    @achakraborti6 жыл бұрын

    First video I see from this channel. Immediately subscribed!

  • @sethagastya
    @sethagastya4 жыл бұрын

    This was an amazing video! Thanks man. Will stay tuned for more!

  • @ativjoshi1049
    @ativjoshi10496 жыл бұрын

    Your explanation is crisp and to the point. Thanks.

  • @double_j3867
    @double_j38676 жыл бұрын

    Subscribed. Very useful -- i'm an applied ML researcher (applying these techniques to real-world problems) so I need a way to quickly "scan" methods and determine what may be useful before diving in-depth. These styles of videos are exactly what I need.

  • @ashokkannan93
    @ashokkannan935 жыл бұрын

    Excellent video!! Probably the best VAE video I saw. Thanks a lot :)

  • @abhinavshaw9112
    @abhinavshaw91126 жыл бұрын

    Hi, I am a Graduate Student at UMass Amherst. I really liked your video, it gave me a lot of ideas. Watching this before reading the paper would really help. Please keep it coming I'll be waiting for more.

  • @SamWestby
    @SamWestby3 жыл бұрын

    Three years later and this still the best VAE video I've seen. Thanks Xander!

  • @MeauxTarabein
    @MeauxTarabein5 жыл бұрын

    Very Helpfully Arxiv! keep the good Quality videos coming

  • @liyiyuan45
    @liyiyuan453 жыл бұрын

    This is sooooooo useful for 2am and you dragged by all the math in the actually paper. Thanks man for the clear explanation!

  • @JakubArnold
    @JakubArnold6 жыл бұрын

    Great explanation on why we actually need the reparameterization trick. Everyone just skims over that and explains the part that mu+var*N(0,1) = N(mu,var), but ignores the part why you need it. Good job!

  • @shivamutreja6427
    @shivamutreja64272 жыл бұрын

    Your videos are absolute crackin for a quick revision before an interview!

  • @famouspeople3499
    @famouspeople34993 жыл бұрын

    Great video, better than many tutor lessons in university, this animation and simplified the things with simple words

  • @AjithKumar-gk7bf
    @AjithKumar-gk7bf5 жыл бұрын

    Just found this channel ... today... one word Brilliant...!!!

  • @antoinesueur9289
    @antoinesueur92896 жыл бұрын

    Great content ! The format and delivery is perfect, hope to see more of these videos :) . Are you planning on doing a video on Capsule Networks in the future ?

  • @ArxivInsights

    @ArxivInsights

    6 жыл бұрын

    More videos are definitely coming, the next one will be on novel state-of-the-art methods in Reinforcement Learning! I don't plan on making a video on Capsule Nets since there is an amazingly good video by Aurélien Géron on that topic and there's no way I can explain it any better than he did, no need to reinvent the wheel :p Here is his video: kzread.info/dash/bejne/ooSCmsZpdZafYJM.html

  • @ejeinstein
    @ejeinstein6 жыл бұрын

    Really really awesome channel!!! Look forward to watching more of your videos!

  • @hitarthk
    @hitarthk5 жыл бұрын

    Absolutely great stuff Arxiv Insights! Subscribed to your videos for life :)

  • @yanfengliu
    @yanfengliu6 жыл бұрын

    This is really good. I like the way you explain things. Thank you for sharing!

  • @vortexZXR
    @vortexZXR4 жыл бұрын

    So many ideas come to mind after watching this video. Well done!

  • @robinranabhat3125
    @robinranabhat31256 жыл бұрын

    Don't you ever stop explaining papers like this. Better than Siraj's video. Just explain the code part a bit longer. And your channel is set.

  • @pablonapan4698

    @pablonapan4698

    6 жыл бұрын

    exactly. show some more code please.

  • @shrangisoni8758

    @shrangisoni8758

    5 жыл бұрын

    Yea we can't really do much until we code and see results ourselves.

  • @pixel7038

    @pixel7038

    4 жыл бұрын

    Siraj has improved his videos and provides more content. Don’t be stuck in the past ;)

  • @gagegolish9306

    @gagegolish9306

    4 жыл бұрын

    @@shrangisoni8758 He's explained the fundamental concepts, you can take those concepts and translate them to code. He shouldn't have to do that for you.

  • @dalchemistt7

    @dalchemistt7

    4 жыл бұрын

    @@pixel7038 Please stop spreading his name. He has faked his way more than enough already. Read more here: twitter.com/AndrewM_Webb/status/1183150368945049605 and here www.reddit.com/r/learnmachinelearning/comments/dheo88/siraj_raval_admits_to_the_plagiarism_claims/ And what really bugs me is not the plagiarism- that's bad and shameful in itself- but the level of stupidity this guys had shown while plagiarizing- "gates" to "doors" and "complex Hilbert space" to "complicated Hilbert space".

  • @superaluis
    @superaluis6 жыл бұрын

    Great channel! Keep up with this awesome project. Already subscribed and going to share this channel with my colleagues.

  • @maxhorowitz-gelb6092
    @maxhorowitz-gelb60926 жыл бұрын

    Wow! Great video. Very concise and easy to understand something quite complex.

  • @kristyleist3318
    @kristyleist33184 жыл бұрын

    This is great! Keep going, we need you! Don't stop making amazing videos like this

  • @TheRohr
    @TheRohr6 жыл бұрын

    Great thanks for the video and the paper explanation! Really, really helpful, keep that paper explanation content!

  • @fktudiablo9579
    @fktudiablo95793 жыл бұрын

    always the best place to have a good overview before diving deeper

  • @matthewbascom
    @matthewbascom4 жыл бұрын

    I like the subtle distinction you made between the disentangled variational auto-encoder versus the normal variational auto-encoder: Changing the first dimension in the latent space of the disentangled version rotates the face while leaving everything else in the image unchanged. But changing the first dimension in the normal version not only rotates the image, but changes other features as well. Thank you. Me gleaning that distinction from Higgins, et al. Beta-VAE Deepmind paper would be unlikely...

  • @joshbrenneman
    @joshbrenneman5 жыл бұрын

    Wow, love your videos. I have not worked with reinforcement learning, but I’d love to hear your analysis of other generative models.

  • @animeshgoyal9583
    @animeshgoyal95835 жыл бұрын

    Simply Amazing! Thanks for sharing this. Absolutely loved it. Hope to see more videos from you :)

  • @davidenders9107
    @davidenders91076 ай бұрын

    Thank you! This was comprehensive and comprehendible.

  • @HeduAI
    @HeduAI5 жыл бұрын

    Amazing explanation to a complicated topic! Thank you so much!!!!

  • @ChocolateMilkCultLeader
    @ChocolateMilkCultLeader2 жыл бұрын

    Shared your work with my followers. Keep making amazing content

  • @abylayamanbayev8403
    @abylayamanbayev8403 Жыл бұрын

    Finally I understood the intuition of sampling from mu and sigma and reparameterization trick. Thanks!

  • @satishbanka
    @satishbanka3 жыл бұрын

    Very good explanation of Variational Autoencoders! Kudos!

  • @sunnybeta_
    @sunnybeta_6 жыл бұрын

    This video suddenly popped up today morning on my home page. Now i know my Sunday will be great. :D

  • @karrikarthik6936
    @karrikarthik69365 жыл бұрын

    Big Fan. Would love to see videos where you breakdown some of the applications using deep learning tools.

  • @AhmedKachkach
    @AhmedKachkach6 жыл бұрын

    Immediate subscribe :) Thanks for this in-depth video. Please keep a format like this in the future (relatively in-depth explanation, to build a real intuition about these techniques).

  • @akshayshrivastava97
    @akshayshrivastava973 жыл бұрын

    Finally, someone who cares their viewers actually get to understand VAEs.

  • @betterbrained
    @betterbrained2 жыл бұрын

    You help so much with my exams, thanks man, subscribed for more high quality stuff!

  • @SlavIvanov
    @SlavIvanov6 жыл бұрын

    This is great! Keep going, we need you!

  • @falsiofalsissimo5313
    @falsiofalsissimo53135 жыл бұрын

    We needed a serious and technical channel about latest findings in DL. That siraj crap is useless. Keep going! Awesome

  • @ck1847
    @ck18472 жыл бұрын

    Thanks, this video clarified many things from the original paper.

  • @dmitrykalashnikov8637
    @dmitrykalashnikov86375 жыл бұрын

    Very useful. Great content. Continue what you're doing. Great job.

  • @SeanLGoldberg
    @SeanLGoldberg6 жыл бұрын

    Great episode. Came here for a good explanation of VAEs, but was blown away when you dug into Beta-VAEs and the Deepmind RL paper. Have you read the group's newest paper "SCAN" on combining Beta-VAEs with symbol representation and manipulation?

  • @ArxivInsights

    @ArxivInsights

    6 жыл бұрын

    Sean Goldberg Haven't had the time yet, it's somewhere in my 658 open chrome tabs though :p

  • @1apiano

    @1apiano

    6 жыл бұрын

    I liked arxiv.org/pdf/1709.05047.pdf more, but the SCAN paper is also cool. BTW compliments for your channel, it's the only Deep Learning channel which is worth following.

  • @user-or7ji5hv8y
    @user-or7ji5hv8y4 жыл бұрын

    Your explanation is so clear.

  • @mrdbourke
    @mrdbourke6 жыл бұрын

    Epic video Xander! I learned a lot from your explanation. Now to try an implement some code!

  • @jfndfiunskj5299
    @jfndfiunskj52996 жыл бұрын

    Very clearly explained. Good job.

  • @muhammaddjamaluddin6125
    @muhammaddjamaluddin61254 жыл бұрын

    Good explanation. Enough and relevant math that support the explanation which we can understand the insight.

  • @TienTaioan
    @TienTaioan6 жыл бұрын

    Very clear explanation. Thank you!

  • @submagr
    @submagr4 жыл бұрын

    Thanks for this video. It gave a nice overall idea about variational auto-encoders.

  • @adityasoni6308
    @adityasoni63086 жыл бұрын

    Amazing description... Need more videos on different things

  • @phattran4858
    @phattran48584 жыл бұрын

    Thank you very much, I was trying to understand it, but it's much easier when I found this video!

  • @emanehab510
    @emanehab5106 жыл бұрын

    Don't stop making amazing videos like this

  • @tamerius1
    @tamerius16 жыл бұрын

    You're explaining this very well! Finally an explanation on an AI technique that's easy to follow and understand. Thank you.

  • @timurbabadjanov9115
    @timurbabadjanov91154 жыл бұрын

    That was a great explanation! Thank you so much!

  • @Golgafrincham
    @Golgafrincham5 жыл бұрын

    Awesome explainations and interesting subjects, keep it up!

  • @md.mottakinchowdhury7898
    @md.mottakinchowdhury78986 жыл бұрын

    This is just good content. Such in depth explanations are what we need in AI community. Great work.

  • @bradknox
    @bradknox3 жыл бұрын

    Great video! I have a minor correction: At 6:14, calling the cursive L a "loss" might be a misnomer, since loss is something we almost always want to minimize, and the formula of (reconstruction likelihood - KL divergence) should be maximized. In fact, the Kingma and Welling paper call that term the "(variational) lower bound on the marginal likelihood of datapoint i", not a loss.

  • @kanglemu6801
    @kanglemu68013 жыл бұрын

    Really love this video! Good job!

  • @DILLIPKUMARSAHOOIITM
    @DILLIPKUMARSAHOOIITM6 жыл бұрын

    Very good explanation. Subscribed to the channel. Looking for more thoughtful videos on cutting edge ML stuff.

  • @PierLim
    @PierLim6 жыл бұрын

    Very good explanation! Thank you man!

  • @garyhuntress6871
    @garyhuntress68716 жыл бұрын

    I really enjoyed that and did learn quite a bit!

  • @user-em7co7ui8y
    @user-em7co7ui8y5 жыл бұрын

    Awesome explanation! Thanks!

  • @thomasomalley4992
    @thomasomalley49926 жыл бұрын

    Wow, great explanation and awesome channel!

  • @user-xd2ul3ym7k
    @user-xd2ul3ym7k5 жыл бұрын

    Nice and clear, good job!

  • @lordsherpaman
    @lordsherpaman5 жыл бұрын

    That was a great explanation, thank you!

Келесі