ICLR 2021 Keynote - "Geometric Deep Learning: The Erlangen Programme of ML" - M Bronstein

Ғылым және технология

Geometric Deep Learning: The Erlangen Programme of ML - ICLR 2021 Keynote by Michael Bronstein (Imperial College London / IDSIA / Twitter)
“Symmetry, as wide or as narrow as you may define its meaning, is one idea by which man through the ages has tried to comprehend and create order, beauty, and perfection.” This poetic definition comes from the great mathematician Hermann Weyl, credited with laying the foundation of our modern theory of the universe. Another great physicist, Philip Anderson, said that "it is only slightly overstating the case to say that physics is the study of symmetry."
In mathematics, symmetry was crucial in the foundation of geometry as we know it in the 19th century. Now it could have a similar impact on another emerging field. Deep Learning success in recent decades is significant - from revolutionising data science to landmark achievements in computer vision, board games, and protein folding. At the same time, a lack of unifying principles makes it is difficult to understand the relations between different neural network architectures resulting in the reinvention and re-branding of the same concepts.
Michael Bronstein is a professor at Imperial College London and Head of Graph ML Research at Twitter, who is working to bring geometric unification of deep learning through the lens of symmetry. In his ICLR 2021 keynote lecture, he presents a common mathematical framework to study the most successful network architectures, giving a constructive procedure to build future machine learning in a principled way that could be applied in new domains such as social science, biology, and drug design.
Based on M. M. Bronstein, J. Bruna, T. Cohen, P. Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges, arXiv:2104.13478, 2021 (arxiv.org/abs/2104.13478)
Accompanying blog post: towardsdatascience.com/geomet...
More information: geometricdeeplearning.com/
Animation: Jakub Makowski

Пікірлер: 167

  • @AbhishekSingh-mz8mb
    @AbhishekSingh-mz8mb3 жыл бұрын

    The presentation quality, content coverage, and animation here is incredibly marvelous! This has certainly set a gold standard for future talks. Thanks a lot for putting this together.

  • @bucketofbarnacles

    @bucketofbarnacles

    3 жыл бұрын

    Couldn’t agree more. Depth, breadth and effectiveness of communication are spot on.

  • @AICoffeeBreak
    @AICoffeeBreak3 жыл бұрын

    What a great keynote, both content-wise and in terms of the visuals. 👏 A good side-product of virtual conferences is certainly the production value of scientific talks going up.

  • @gracechang7947
    @gracechang79472 жыл бұрын

    Incredible, really enjoyed this keynote. Agree, one of the best presentations on ML I’ve seen yet. I’m really happy to see the emphasis on clarity to a general audience with such well-crafted illustrations of concepts.

  • @ehtax
    @ehtax3 жыл бұрын

    Presentation mastery! You managed to boil things down to the most salient intuitions, all the while covering such a wide breadth of topics! This has me amped to dive into your papers (im in fmri neuroscience, where graph-based predictive modelling has been mostly ineffectual thusfar)

  • @tst3rt
    @tst3rt2 жыл бұрын

    Спасибо, Михаил! Одна из лучших презентаций, которые я видел.

  • @tienphammanh4224
    @tienphammanh42243 жыл бұрын

    This talk is so amazing. I really like your interpretation of mathematical formulas, very clearly. Thanks for your great work. Hope you make more videos like this. One more time, thank you very much.

  • @Fordance100
    @Fordance1003 жыл бұрын

    Very interesting perspectives on deep learning and seamless transition from one concept to another. Truly a master piece of scientific presentation. Thank you so much for posting it.

  • @zorqis
    @zorqis3 жыл бұрын

    This was deeply thought provoking and wonderfully inspiring.

  • @schumachersbatman5094
    @schumachersbatman50943 жыл бұрын

    This is the best presentation on machine learning I've ever seen. So enjoyable.

  • @MachineLearningStreetTalk
    @MachineLearningStreetTalk3 жыл бұрын

    Amazing stuff! Hope we can interview Prof. Bronstein on our show soon 😀

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    would be honored

  • @LovroVrcek
    @LovroVrcek3 жыл бұрын

    This should be a gold standard of keynote talks. Amazing! 👏

  • @kosolapovlev6029
    @kosolapovlev60293 жыл бұрын

    This is literally the best presentation about machine learning I have ever seen. Thank you for your marvelous work!

  • @Jacob011

    @Jacob011

    2 жыл бұрын

    It is very intriguing research and graphically well presented. I wonder what relationships are there between this unifying geometric perspective of deep learning and the random finite sets (stochastic geometry, poison point processes), which are now the rave in the multi-object tracking community. This presentation is also slightly infuriating in that it goes over very deep concepts very fast. Regardless though, amazing work!

  • @youcefouadjer8057
    @youcefouadjer80573 жыл бұрын

    The incredible Michael Bronstein is on KZread !! This is Awesome

  • @3ss3ns3
    @3ss3ns32 жыл бұрын

    very good coverage. thank you, Prof. Bronstein

  • @adrianharo6586
    @adrianharo65863 жыл бұрын

    I wish I could understand all the details, but my education only takes me so far understanding the concepts you're going over. I am a newbie ML enthusiast. I really do appreciate the animation, it is nice to follow it.

  • @ahmadzobairsurosh5832
    @ahmadzobairsurosh5832 Жыл бұрын

    Absolutely Amazing Prof Bronstein! Thank you for such an amazing piece of content.

  • @raghavamorusupalli7557
    @raghavamorusupalli75572 жыл бұрын

    It takes a semester for us to comprehend this marathon talk, Sir. Great visionary talk. Thank you Sir

  • @benganot4363
    @benganot43633 жыл бұрын

    As a computer science student now preparing for his ML course exam. I was just blown away by how all machine learning algorithms are related. Beautiful, stunning work.

  • @LukePluto
    @LukePluto2 жыл бұрын

    This is amazing. I hope you make more videos like this again!

  • @VitorMeriat
    @VitorMeriat3 жыл бұрын

    Geometric Deep Learning Grids, Groups, Graphs, Geodesics, and Gauges is of great importance for my master's degree. Great presentation, is an honor.

  • @sabawalid
    @sabawalid2 жыл бұрын

    Great work... this has the chance to advance DL considerably, especially detecting "intrinsic features" which will solve many existing problems This is real science !!! Thumbs up!

  • @phillipyu6260
    @phillipyu62603 жыл бұрын

    This inspires me to continue my education. My brain is itching to learn more!

  • @renegadephalanx
    @renegadephalanx3 жыл бұрын

    Great, concise, and very explanatory presentation. Thank you very much for uploading this content.

  • @thevirtualguy5074
    @thevirtualguy50743 жыл бұрын

    This is EPIC! looking forward to more of this great material.

  • @vishalmishra3046
    @vishalmishra30462 жыл бұрын

    This approach to Geometric Neural Nets is like a potential Nobel prize winning grand unification theory (GUT) unifying all the neural net architectures from ANN, CNN, RNN, Graph-NN, Message Passing (MP-NNs) neural nets and Transformers (Attention Neural Nets). Wonderful video !! Just like M-Theory when there is too much innovation accumulating over time, a simplifier needs to be born who can merge and unify all of them into a single more general purpose abstraction.

  • @icanfast
    @icanfast2 жыл бұрын

    i was in awe to see how underlying maths unifies DL techniques. Daresay community NEEDS a similar but in-depth deconstruction of particular topics. There are a lot of knowledgeable people in the comments, someone please make it happen

  • @HtHt-in7vt
    @HtHt-in7vt2 жыл бұрын

    Well done! Clear and visual! Please more like that! Thanks a lot!

  • @fulcobohle4576
    @fulcobohle45762 жыл бұрын

    I was amazed by your presentation, good job. But what amazed me was that I was able to understand in detail everything you explained. 35 years ago I studied physics and mathematics and learned all aspects of what you told in this video without ever realizing it could be applied to AI as well. Like you I was confused about the why of convolution, thanks for giving me the light !

  • @ashra.academy
    @ashra.academy2 жыл бұрын

    I'm in love with this presentation format! Would you consider sharing the Illustrator and After Effect project files? I'd like to learn how to do this and have no clue where to start!

  • @pianoconlatte
    @pianoconlatte2 жыл бұрын

    Beautiful presentation. Got some ideas to test.Thank you.

  • @khuongnguyenduy2156
    @khuongnguyenduy21563 жыл бұрын

    Thank you very much for your great talk!

  • @MarianaViale
    @MarianaViale2 жыл бұрын

    Thank you for this great presentation and for sharing it with the common public.

  • @max477
    @max4773 жыл бұрын

    I feel sad that I left this field for financial reason. But I keep watching these videos

  • @jonathansum9084
    @jonathansum90843 жыл бұрын

    Thank you for uploading. I hope it will talk about the coding part too.

  • @ashwindesilva4781
    @ashwindesilva47812 жыл бұрын

    Such an amazing lecture! Thank you very much :)

  • @luisleal4169
    @luisleal41692 жыл бұрын

    Wow! this is an excelent presentation, I guess your classes are something like this, and your students are very lucky to have you as a professor.

  • @simpl51
    @simpl513 жыл бұрын

    Thank you so much for this. After Sunday lunch, Idling through youtube, i was dragged down a nD rabbit hole, through some maths and psycology history fo some hary transformations of a non-trivial representation into a managable ones, and how they can improve the lives of astronomers, computer gamers, and pharmacologists,. How mapphg foods and drugs could alleviate diseases;. How computers could troll through posts and comments to find a small subset of interesting ones.. Even youtube itself joined in, and removed adverts, brexit rants, music, and chess blogs from my starter screen. What a great life you lead!

  • @asnaeb2
    @asnaeb23 жыл бұрын

    Very nice animations make it a lot easier to follow. Thanks!

  • @smcg2490
    @smcg2490 Жыл бұрын

    Wow. Just. Wow. The quality of this presentation is incredible. The animations enabled me to grasp concepts (almost) instantly. So incredibly helpful for my current paper. Thank you ever so much for the money, time, and effort it took to produce a video of such exceptional quality.

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    Жыл бұрын

    Thank you. Such comments are the best motivation to continue doing more!

  • @amirleshem6720
    @amirleshem67202 жыл бұрын

    Amazing. I'm speechless.

  • @TL-fe9si
    @TL-fe9si2 жыл бұрын

    Thank you! amazing presentation!!! I giggled a little when seeing 2:40

  • @MDLI
    @MDLI3 жыл бұрын

    Wow, you took it to the next level! Super informative and impressive.

  • @dawithailu3439
    @dawithailu34392 жыл бұрын

    I wasn't sure at first as to how you wanted to connect the different geometries with deep learning , but as the video went on, I could see what you meant. And now, I am thinking about how it can be applied in emotion classification project I'm interested in. Thank you for the general insight, It would be incredibly awesome if you can attach some git works.

  • @MrAceman82
    @MrAceman823 жыл бұрын

    I must admit, I came to this link accidentally. The presentation is a master piece. Keep it going. Following.

  • @hrishikeshkhaladkar4963
    @hrishikeshkhaladkar49632 жыл бұрын

    This is amazing sir..Hopefully this will motivate the student community to take up mathematics very seriously

  • @krishnaaditya2086
    @krishnaaditya20863 жыл бұрын

    Awesome Thanks!

  • @vi5hnupradeep
    @vi5hnupradeep3 жыл бұрын

    Just wow 💯 ; this is inspiring me to learn more ,. Amazing presentation 💫

  • @peggy767
    @peggy7672 жыл бұрын

    Such an inspiring presentation!

  • @rendermanpro
    @rendermanpro2 жыл бұрын

    Presentation quality is stuning

  • @TheMrleo2107
    @TheMrleo21073 жыл бұрын

    A great presentation professor. Reminds me of 3blue1brown

  • @florianro.9185
    @florianro.91852 жыл бұрын

    Absolutely great presentation! What software was used to create these animations? :) Thanks

  • @amirhosseindaraie5622
    @amirhosseindaraie56223 жыл бұрын

    This was wonderful!!!!!!!

  • @sandeepvk
    @sandeepvk2 жыл бұрын

    I am quite excited about this field. Traditionally the innovation in biotech engineering was hampered by ethical concerns. With this technique we can quickly innovate without any political ramification. This is quite akin to the growth of internet itself

  • @rigidrobot
    @rigidrobot2 жыл бұрын

    Is one of the possible domains of GDL going to be in any instance of a dynamic system? For instance not just proteins but interactions between molecular pathways? Or meme propagation networks?

  • @robinranabhat3125
    @robinranabhat31253 жыл бұрын

    Now this was enlightening !

  • @user-kb6je7bx9e
    @user-kb6je7bx9e2 жыл бұрын

    This is amazing presentation 👍👍👍

  • @a_sobah
    @a_sobah3 жыл бұрын

    Very interesting I all ways have that question is there a way to indefinitely transformation on deeplearning this video shows how it's done thank you like to more on this topic but it's hard for me to understand all those mathematics.

  • @chaoyang1945
    @chaoyang19453 жыл бұрын

    Great talk!!!!

  • @Arkonis1
    @Arkonis13 жыл бұрын

    The introduction reminds me talks from S. Mallat where he was already in 2012 showing in one hand the underlying symmetry invariance that we have in his wavelett scattering system and on the other hand the analogy of this system with deep CNN. And concluding that deep learning architecture might learn symmetry groups invariance like learning the groups of cats, dogs, tables etc.. I like very much this group theory approach, which is not often discussed in literature so far

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    Indeed we cite Mallat in the book - his paper with Joan Bruna on scattering network established that CNNs are not only shift-equivariant but also approximately equivariant to smooth deformations

  • @haitham973
    @haitham9733 жыл бұрын

    Super cool talk!!

  • @user-jh8rd2pm3z
    @user-jh8rd2pm3z2 жыл бұрын

    This is really amazing!

  • @ElaprendizdeSalomon
    @ElaprendizdeSalomon7 ай бұрын

    wonderful work.

  • @LucasRolimm
    @LucasRolimm3 жыл бұрын

    Master piece!

  • @madhavpr
    @madhavpr2 жыл бұрын

    This is one of the most beautiful presentations I have ever seen in my life. I'll be honest here- I did not understand much, but I'm truly inspired to learn the material. Professor Bronstein, would a deep learning / signal processing background be enough to pick up this material?

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    2 жыл бұрын

    I would give a biased response, but probably our forthcoming book we are currently writing (a preview is available here: arxiv.org/abs/2104.13478)

  • @dihuang9849
    @dihuang98492 жыл бұрын

    Awesome!

  • @r0lisz
    @r0lisz3 жыл бұрын

    Great talk! And outstanding visuals! How were they made?

  • @AndyTutify

    @AndyTutify

    3 жыл бұрын

    You could make this in After Effects

  • @VictorBanerjeeF
    @VictorBanerjeeF3 жыл бұрын

    Love at first sight... ❤️

  • @Hassan-se3vx
    @Hassan-se3vx2 жыл бұрын

    Very nice presentation

  • @xbronn
    @xbronn2 жыл бұрын

    omfg, wow. what a presentation!

  • @roomo7time
    @roomo7time2 жыл бұрын

    absolute gold

  • @stimpacks
    @stimpacks2 жыл бұрын

    OK, I now need a Hinton, Bengio, LeCunn & Schmidthuber print. In an antique frame.

  • @LukeVilent
    @LukeVilent2 жыл бұрын

    Oh yeah, RealSense, I've been working with them in image recognition, trying to build something similar to Complex Yolo, but in a more engineering way. However, the quality was not suited for the harsh conditions we were exposing the devices to (pig stall). It was also the time when the first extensive neuronal network libraries became available, and I've said that in a few years the tech calibration of the camera will be just replaced by a neural network. And, broadly speaking, that's what drives my current research.

  • @3laserbeam3
    @3laserbeam33 жыл бұрын

    Damn! That's awesome! As a side note, may I ask what was used to create the visuals and animations for this talk? They are gorgeous!

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    Adobe AE and two months of work of two professional designers

  • @3laserbeam3

    @3laserbeam3

    3 жыл бұрын

    @@MichaelBronsteinGDL That would have been my guess, professional designers involved. Thanks!

  • @MrAceman82

    @MrAceman82

    3 жыл бұрын

    @@MichaelBronsteinGDL Great animations, and thank you for your efforts to share this valuable knowledge.

  • @JousefM
    @JousefM2 жыл бұрын

    Wow, that's so dope!!! Thanks for this great production quality and delivery Michael! Btw, would love to have you on my podcast talking about GDL!

  • @EfraM83
    @EfraM832 жыл бұрын

    interesting.... I'm working on the same thing independently.... I believe this is ultimately the theory of everything.

  • @pafloxyq
    @pafloxyq3 жыл бұрын

    A very cool presentation, just wanted to ask if the scale transformation described at 09:31 has anything to do with renormalization groups methods in physics ?

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    I don’t see an immediate connection

  • @xinformatics

    @xinformatics

    2 жыл бұрын

    i get it what you say; good point imo

  • @user-ce1by6zy7b
    @user-ce1by6zy7b2 жыл бұрын

    Only got here from other videos on the topic. Nice presentation, one that assumes a bit more linear algebra and group theory fundamentals (but indeed one only needs the very basics of those fields + basics of analysis to follow the concepts in ML/DL), but gets a bit more into actual details compared to other videos I have watched on the same topic, which I appreciated. If only there weren't so many self-promoting plugs all over the place throughout the video, it gave me the impression that the actual science on the video served as an instrument for own work promotion a bit too much. I guess it might be a cultural trait of the field and this is how things work, but from what I gathered from the comments, active or former researchers in the field (I don't qualify as such) already know not only you, but your work as well (which I have absolutely no doubt to assume that is indeed very noteworthy), already prior to the video. Subscribed.

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    2 жыл бұрын

    I think invited speakers are invited exactly because of their expertise, and it is expected to talk about own work (hence the "self-promoting plugs", which are some of the first works in the field that we did with students and collaborators). In the book we show a more balanced overview, however for the video I chose those works I relate to more.

  • @jungjunk1662
    @jungjunk16622 жыл бұрын

    This presentation is as great as the talk itself. What software did you use to create the presentation graphics?

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    2 жыл бұрын

    was done by professional designers. photoshop/illustrator/after effects

  • @harriehausenman8623
    @harriehausenman86233 жыл бұрын

    Thank you for the great video. I wonder what Stephen Wolfram thinks about this ;-)

  • @RuoyangYao
    @RuoyangYao2 жыл бұрын

    This is amazing.

  • @lennylenny7320
    @lennylenny73202 жыл бұрын

    awesome!!

  • @chrissgouros7282
    @chrissgouros72823 жыл бұрын

    ΕΚΠΛΗΚΤΙΚΟΣ!!

  • @fengliu7904
    @fengliu79042 жыл бұрын

    Imagine how much time the presenter has spent preparing this presentation.

  • @louerleseigneur4532
    @louerleseigneur45322 жыл бұрын

    Thanks

  • @user-mp9gj4mg6z
    @user-mp9gj4mg6z3 жыл бұрын

    Thanks for the video. I wanted to know more about this view of machine learning.

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    Check our proto-book on which the talk is based: arxiv.org/abs/2104.13478

  • @user-mp9gj4mg6z

    @user-mp9gj4mg6z

    3 жыл бұрын

    @@MichaelBronsteinGDL thanks

  • @federicocarrone512
    @federicocarrone5122 жыл бұрын

    this is amazing

  • @kirekadan
    @kirekadan2 жыл бұрын

    Great presentation. Can you tell me how the software you use to animate the graphs?

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    2 жыл бұрын

    AfterEffects

  • @laurencevanhelsuwe3052
    @laurencevanhelsuwe30522 жыл бұрын

    My old math teacher would break out in a sweat of disbelief seeing that higher mathematics can be used to recognise cats !

  • @sudarshanregmi14
    @sudarshanregmi143 жыл бұрын

    Nice!

  • @imalive404
    @imalive4042 жыл бұрын

    Full fledged AR and VR products are gonna be launched soon is one of the takes. Metaverse is here

  • @outruller
    @outruller2 жыл бұрын

    Oh. My. God. It a shame that I am too dumb to deeply understand everything that was said, nevertheless even what I did get is astonishingly fascinating! I so regret not learning harder in my university days, may be I would have had a chance to work on something this impactful and motivating.

  • @pratikdeshpande3258
    @pratikdeshpande32582 жыл бұрын

    Excellent generalisation of deep learning. I can see Linear Algebra, Graph theory, Group theory and many other math branches intersecting with physics, computer graphics and biology. This is truly a gem of ML. BTW, what's on the y-axis of this graph at 18:58 ?

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    2 жыл бұрын

    The task is regressing the penalized water-octanol partition coefficient (logP) on molecules from the ZINC dataset. Y-axis shows the testing Mean Absolute Error.

  • @TheAIEpiphany
    @TheAIEpiphany2 жыл бұрын

    It's year 2030. MLPs are SOTA on all domains imaginable to human mind. MLP AGI whispers: Michael didn't mention me in his ICLR keynote. Paperclips.

  • @adrianharo6586
    @adrianharo65863 жыл бұрын

    Where can I find more information on the project that helps classify the molecules on plant based foods??

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    Here is a blog post: towardsdatascience.com/hyperfoods-9582e5d9a8e4?sk=d20fe73c7d9ecb62dd3d391a44d4ef7f

  • @priyamdey3298

    @priyamdey3298

    3 жыл бұрын

    My mind was blown away when I saw that even food preparation can be represented as a computational graph with cooking transformations as edges and optimize to maximally preserve the anti-cancer effect 🙌.

  • @georgeb8637
    @georgeb86373 ай бұрын

    28:38 - 3D sensor to capture face - 10 years ago - Intel integrated 3D sensor into their product 30:17 - we don’t need a 3D sensor now - we can use 2D video + geometric decoder that reconstructs a 3D shape 36:50 - tea, cabbage, celery, sage

  • @TriPham-xd9wk
    @TriPham-xd9wk2 жыл бұрын

    Time base from data to force altering lead to transformation and amphomorism. Like water it remain water in different temperature so it survival all economic, political, and religious condition and remain an kind, compassionate, and creative wise human

  • @fredxu9826
    @fredxu98263 жыл бұрын

    How is this presentation created (tools)? Would love to follow the path of Dr. Bronstein and start creating presentations like this one.

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    3 жыл бұрын

    That was a (titanic) work of Jakub Makowski with Adobe AE. Nearly two month.

  • @fredxu9826

    @fredxu9826

    3 жыл бұрын

    @@MichaelBronsteinGDL wow I guess I will endeavor on the art-side of the project after my theory is worth the effort :)

  • @user-bm4yf6td7d
    @user-bm4yf6td7d2 жыл бұрын

    Михаил Бронштейн наверное русскоязычный? Ваша фамилия как то связана с тем, что т9 её подсказывает? И что вы думаете о модели сегрегации Шеллинга?

  • @MichaelBronsteinGDL

    @MichaelBronsteinGDL

    2 жыл бұрын

    Да, русскоязычный (родился в России но вырос в Израиле). Никогда не имел дело с этой моделью.

  • @YangQuanChen
    @YangQuanChen3 жыл бұрын

    I heard "long range interaction" interesting

  • @saulberardo5826
    @saulberardo58263 жыл бұрын

    👏👏👏

Келесі