Shannon Entropy and Information Gain

Ғылым және технология

Announcement: New Book by Luis Serrano! Grokking Machine Learning. bit.ly/grokkingML
40% discount code: serranoyt
Accompanying blog post: / 5810d35d54b4
0:00 Shannon Entropy and Information Gain
2:22 What ball will we pick?
4:33 Quiz
5:06 Question
5:14 Game
7:17 Probability of Winning
7:45 Products
11:00 What if there are more classes?
12:34 Sequence 2
13:44 Sequence 3
14:57 Naive Approach
15:34 Sequence 1
19:44 General Formula

Пікірлер: 327

  • @sphengle
    @sphengle5 жыл бұрын

    This was exactly the baby step I needed to get me on my way with entropy. Far too many people try to explain it by going straight to the equation. There's no intuition in that. Brilliant explanation. I finally understand it.

  • @jankinsics

    @jankinsics

    4 жыл бұрын

    Sean Walsh feel the same way.

  • @freemanguess8634
    @freemanguess86345 жыл бұрын

    With great knowledge comes low entropy

  • @SerranoAcademy

    @SerranoAcademy

    5 жыл бұрын

    Hahaaa, love it!!!

  • @fantomraja9137

    @fantomraja9137

    4 жыл бұрын

    lol

  • @hyperduality2838

    @hyperduality2838

    3 жыл бұрын

    @@SerranoAcademy Repetition (redundancy) is dual to variation -- music. Certainty is dual to uncertainty -- the Heisenberg certainty/uncertainty principle. Syntropy (prediction) is dual to increasing entropy -- the 4th law of thermodynamics. Randomness (entropy) is dual to order (predictability) -- "Always two there are" -- Yoda.

  • @MrofficialC

    @MrofficialC

    2 жыл бұрын

    And low entropy is easier to rig

  • @lani0

    @lani0

    2 жыл бұрын

    You win

  • @AlexMcClung97
    @AlexMcClung976 жыл бұрын

    Excellent explanation, very clear and concise! I have always pondered the significance of the log in cross-entropy loss function. The explanation (particularly: "products are small and volatile, sums are good") completely clears this up.

  • @user-or7ji5hv8y
    @user-or7ji5hv8y3 жыл бұрын

    how does one make something so complicated into something so intuitive that others can finally see the picture. your explanation itself is an amazing feat.

  • @carnivalwrestler
    @carnivalwrestler5 жыл бұрын

    Luis, you are such an incredibly gifted teacher and so meticulous in your explanations. Thank you for your hard work.

  • @mau_lopez
    @mau_lopez6 жыл бұрын

    What a great explanation ! I wish I had a teacher like you Luis, everything wold be way easier ! Thanks a lot

  • @effemmkay
    @effemmkay3 жыл бұрын

    I have been scared of delving into entropy in detail for so long because the first time I studied it, it wasn’t a good experience. All I want to say is THANK YOU!!!!!! I should have been supplementing the udacity ND lesson videos with these since the beginning.

  • @MatheusSilva-dragon
    @MatheusSilva-dragon5 жыл бұрын

    Wow, thank you, man. I needed that information! There are many ways to teach the same stuff! That number of question stuff is great! It's good to have more than one way to measure something!

  • @dyutinrobin
    @dyutinrobinАй бұрын

    Thank you so much. This was the only video in youtube that clarified all my doubts regarding the topic of entropy.

  • @ketlebelninja
    @ketlebelninja5 жыл бұрын

    This was one of the best explanations on entropy. Thanks

  • @shekelboi
    @shekelboi5 жыл бұрын

    Thanks a lot Luis, just had an exam about this Wednesday and your video helped me a lot to understand the whole concept.

  • @sasthra3159
    @sasthra31592 жыл бұрын

    Great clarity. Have never got this idea about the Shannon Entropy. Thank you. Great work!

  • @msctube45
    @msctube453 жыл бұрын

    I needed this video to get me up to speed on entropy. Great job Luis!

  • @123liveo
    @123liveo5 жыл бұрын

    2nd time I found this video and loved it both times. Much better description than the prof at the uni I am at!!!

  • @therealsachin
    @therealsachin6 жыл бұрын

    The best explanation about Shannon entropy that I have ever heard. Thanks!

  • @NoOne-uz4vs
    @NoOne-uz4vs4 жыл бұрын

    I'm studying Decision Tree (Machine Learning Algorithm) and it uses Entropy to efficiently build the tree. I finally understand the details. Thank you!!

  • @sdsa007
    @sdsa00711 ай бұрын

    Wow! Awesome, so books and encyclopedias and biographies of Shannon to understand what you just clearly explained! Thank You!

  • @patricklemaire225
    @patricklemaire2256 жыл бұрын

    Great video! Now I understand what Claude Shannon discovered and how useful and essential maths are in Computer Science.

  • @SenhorMsandiFelipe
    @SenhorMsandiFelipe2 жыл бұрын

    Gracias. Muito claro Senhor. I have been struggling to wrap my head around this and you just made it easy. Thank you.

  • @eprabhat
    @eprabhat6 жыл бұрын

    Luis, You have a great way of explaining. At times , I like your videos more than even some highly rated professors

  • @drakZes
    @drakZes5 жыл бұрын

    Great work. Compared to my textbook you explained it 100 times better, Thank you.

  • @Vuvuzella16
    @Vuvuzella164 жыл бұрын

    This video is helping to keep me floating in my Data Science course; thank you so much for your time!

  • @SixStringTheory6
    @SixStringTheory66 жыл бұрын

    Wow ..... I wish more people could teach like you this is so insightful

  • @Asli_Dexter
    @Asli_Dexter6 жыл бұрын

    i wish i had this lecture during college examination.....still it's nice to finally understand the intuition behind the formulas i already knew.

  • @pixboi

    @pixboi

    5 жыл бұрын

    Teaching should be like this, from practice to theory - no the other way around!

  • @poxyu_was_here
    @poxyu_was_here6 жыл бұрын

    Easy and Great explanation! Thank you very much, Luis

  • @jackallread
    @jackallread4 ай бұрын

    Thanks for the relationship between knowledge and entropy, that was very helpful. Your explanation of statistics is also good! Though, I am only half way through the video at this point, I will finish it! Thanks

  • @elmoreglidingclub3030
    @elmoreglidingclub30303 жыл бұрын

    Excellent! Great explanation. Enjoyable video (except YT’s endless, annoying ads). Thank you for composing and posting.

  • @bismeetsingh352
    @bismeetsingh3524 жыл бұрын

    That was highly intuitive, thank you, sir, I appreciate the effort behind this.

  • @haimmadmon3531
    @haimmadmon35314 жыл бұрын

    Very good explanation - hope to hear more of your videos

  • @paulstevenconyngham7880
    @paulstevenconyngham78805 жыл бұрын

    this is a really great explanation, thanks so much for sharing mate!

  • @mehmetzekeriyayangn3782
    @mehmetzekeriyayangn37824 жыл бұрын

    You are the best.Such a great explanation.Better than lots of text books.

  • @moshad2002
    @moshad20025 жыл бұрын

    amazing explanation!! very clear and articulate. Thank you!

  • @user-xo2vg9om4v
    @user-xo2vg9om4v4 жыл бұрын

    It's very helpful for me to introduce the concept of entropy to students. Thank you for your clear presentation of entropy.

  • @kani8329
    @kani83293 жыл бұрын

    Very nice video. Insightful, inutuitive and very well explained. Thank you!

  • @pkittali
    @pkittali6 жыл бұрын

    Lovely explanation...Superb

  • @RobertLugg
    @RobertLugg5 жыл бұрын

    I have learned so much from your teaching. Thank you.

  • @yikenicolezhang
    @yikenicolezhang3 жыл бұрын

    Thank you so much for explaining this concept!

  • @rolfbecker4512
    @rolfbecker45123 жыл бұрын

    Thank you very much for this beautiful and clear explanation!

  • @karinasakurai9867
    @karinasakurai98674 жыл бұрын

    Brilliant lecture! I learn so much with this explanation. Thanks from Brazil :)

  • @scherwinn
    @scherwinn5 жыл бұрын

    Very clever explanation of mighty ENTROPY.

  • @hanaizdihar4368
    @hanaizdihar43683 жыл бұрын

    What a great explanation! And so i subscribed😊

  • @victorialeigh2726
    @victorialeigh27262 жыл бұрын

    Hola Luis, estupendo, espectacular, excelente!

  • @amperro
    @amperro3 жыл бұрын

    I watched it straight through. Very good.

  • @krupalshah1487
    @krupalshah14875 жыл бұрын

    You explained it really very well. Thank you !

  • @saadjabrani7781
    @saadjabrani77815 жыл бұрын

    easy and thorough explanation, thank you !

  • @rajudey1673
    @rajudey16733 жыл бұрын

    Really, you have given us outstanding information.

  • @MrMaipeople
    @MrMaipeople5 жыл бұрын

    Great and Superb. Thank you so much for the most clearer explanation.

  • @ishayauperelman2926
    @ishayauperelman29265 жыл бұрын

    Great video, simple and to the point.

  • @symnshah
    @symnshah5 жыл бұрын

    Nice explanation. Thank you so much for this video.

  • @nijunicholas631
    @nijunicholas6314 жыл бұрын

    Thanks..Got the intuition behind Entropy

  • @capuleto126
    @capuleto1266 жыл бұрын

    Very well explained, thanks Luis

  • @user-or7ji5hv8y
    @user-or7ji5hv8y4 жыл бұрын

    wow, another great and insightful presentation . really helps to build intuition

  • @logosfabula
    @logosfabula6 жыл бұрын

    Luis, you really are a great communicator. Looking forward to your other explanations.

  • @xThomas1995
    @xThomas19954 жыл бұрын

    Thank you for the very good video. Easiest to understand so far.

  • @sirishachanumolu3723
    @sirishachanumolu37235 жыл бұрын

    Best explanation of entropy. Thanks.

  • @user-wi1rj4iw9y
    @user-wi1rj4iw9y3 жыл бұрын

    十分谢谢! Thank you very much, Luis.

  • @tilugulilwa
    @tilugulilwa3 жыл бұрын

    Superb step by step explanation

  • @JaroslawSchaller
    @JaroslawSchaller5 жыл бұрын

    Beautiful explanation, thank you!

  • @miguelfernandosilvacastron3279
    @miguelfernandosilvacastron32794 жыл бұрын

    Thank you. Nice, concise explanation.

  • @hanaelkhalifa2630
    @hanaelkhalifa26303 жыл бұрын

    Thank you for excellent explanation of entropy concept first... Then reach to final equation step-by-step it is really good and simple way

  • @MostafaMASLOUHI
    @MostafaMASLOUHI3 жыл бұрын

    Thank you very much. Very nice explanation.

  • @adonis100
    @adonis1005 жыл бұрын

    Fascinating. Excellent Job

  • @yhat314
    @yhat3146 жыл бұрын

    Lovely job Luis! Very very good!

  • @generationgap416
    @generationgap4162 жыл бұрын

    Confession: I was a math kiddy; I know to use it but I often missed the deeper meaning and intuition. Your videos are turning me into a math hacker.

  • @shakeelurrahman1846
    @shakeelurrahman1846 Жыл бұрын

    thanks a lot for such a beautiful explanation..!

  • @jahngad
    @jahngad6 жыл бұрын

    great , It helped me alot . Thank you Luis

  • @hoarong1993
    @hoarong19936 жыл бұрын

    You are awesome!. I hope you will make more videos like this

  • @smitomborah467
    @smitomborah4675 жыл бұрын

    thanks a lot, Luis. This was helpful.

  • @YouUndeground
    @YouUndeground5 жыл бұрын

    It was a great explanation! Thank you.

  • @RenanCostaYT
    @RenanCostaYT4 жыл бұрын

    Great explanation, greetings from Brazil!

  • @TheZilizopendwa
    @TheZilizopendwa2 жыл бұрын

    Excellent presentation for an otherwise complex concept.

  • @francismcguire6884
    @francismcguire68845 жыл бұрын

    Best instructor there is! Thanks

  • @carlitos5336
    @carlitos53363 жыл бұрын

    Excelente explicación! Gracias por compartirla.

  • @scherwinn
    @scherwinn6 жыл бұрын

    Mr. Luis Serrano III great job in Neural Network and Claude Shannon Entropy.

  • @Omsip123
    @Omsip1235 ай бұрын

    Very well explained, thank you

  • @marakhider7069
    @marakhider70696 жыл бұрын

    Nice explanations! Thank you.

  • @kleberloayza7839
    @kleberloayza78394 жыл бұрын

    hi Luis, nice to meet you, I am reading the book of Deep learning of Ian Godfellow, and I needed to view your video for understand the chapter, 3.13 information theory. thanks very much.

  • @riderblack6401
    @riderblack64016 жыл бұрын

    Thank you so much! Luis!

  • @MinhLe-xk5rm
    @MinhLe-xk5rm4 жыл бұрын

    Amazing explanation, thanks sir!

  • @pooltablemechanic
    @pooltablemechanic5 жыл бұрын

    Great, clear explanation! Thanks

  • @hcgaron
    @hcgaron5 жыл бұрын

    wonderful video. Thanks!

  • @LAChinthaka
    @LAChinthaka4 жыл бұрын

    Thanks for the great explanation.

  • @kasraamanat5453
    @kasraamanat54532 жыл бұрын

    best, as always ❤️ thank you Luis❤️

  • @Bvic3
    @Bvic35 жыл бұрын

    At 13:44 it's not 0.000488 but 0.00006103515 ! There is a computation error. The entropy is correct, 1.75.

  • @SerranoAcademy

    @SerranoAcademy

    4 жыл бұрын

    Thank you for the correction! Yes, you're right.

  • @markbordelon1601
    @markbordelon16015 жыл бұрын

    Perfect explanation of this

  • @mortezaaslanzadeh3735
    @mortezaaslanzadeh37353 жыл бұрын

    Awesome explanation, you rock

  • @zinahe
    @zinahe4 жыл бұрын

    Thank you very much for sharing. I found it very helpful.

  • @nehabetgeri4490
    @nehabetgeri44906 жыл бұрын

    Very well explained. Thank you.

  • @vandhanagopalakrishnan9401
    @vandhanagopalakrishnan94016 жыл бұрын

    Thank You for the great explanation.

  • @maryamzolnoori1621
    @maryamzolnoori16215 жыл бұрын

    Awesome video, very well explained.

  • @clarakorfmacher7394
    @clarakorfmacher73943 жыл бұрын

    Great Video! I really liked the intuitive approach. My professors was waaaay messier.

  • @basmairtahi3225
    @basmairtahi32255 жыл бұрын

    Super perfect explanation ! thanks

  • @kingshukbanerjee748
    @kingshukbanerjee7485 жыл бұрын

    very lucid explanation - excellent, intuitive build-up to Shannon's theorem from scratch

  • @arunkumarvediappan9062
    @arunkumarvediappan90625 жыл бұрын

    awsome explanation on the formula sir

  • @Illumarnati
    @Illumarnati3 жыл бұрын

    Very nicely done.

  • @abhishek_sengupta
    @abhishek_sengupta3 жыл бұрын

    Awesome explanation!!

  • @Dennis12869
    @Dennis128694 жыл бұрын

    Best explanation I found so far

  • @andrashorvath2411
    @andrashorvath24112 жыл бұрын

    Fantastic work.

  • @emrahyener402
    @emrahyener4022 жыл бұрын

    Thanks for this perfect explanation 👏👏👏👍

  • @mulangonando2942
    @mulangonando2942 Жыл бұрын

    I love the explanation of the negative sign in the Entropy Equation many people wonder

Келесі