Shannon Entropy and Information Gain
Ғылым және технология
Announcement: New Book by Luis Serrano! Grokking Machine Learning. bit.ly/grokkingML
40% discount code: serranoyt
Accompanying blog post: / 5810d35d54b4
0:00 Shannon Entropy and Information Gain
2:22 What ball will we pick?
4:33 Quiz
5:06 Question
5:14 Game
7:17 Probability of Winning
7:45 Products
11:00 What if there are more classes?
12:34 Sequence 2
13:44 Sequence 3
14:57 Naive Approach
15:34 Sequence 1
19:44 General Formula
Пікірлер: 327
This was exactly the baby step I needed to get me on my way with entropy. Far too many people try to explain it by going straight to the equation. There's no intuition in that. Brilliant explanation. I finally understand it.
@jankinsics
4 жыл бұрын
Sean Walsh feel the same way.
With great knowledge comes low entropy
@SerranoAcademy
5 жыл бұрын
Hahaaa, love it!!!
@fantomraja9137
4 жыл бұрын
lol
@hyperduality2838
3 жыл бұрын
@@SerranoAcademy Repetition (redundancy) is dual to variation -- music. Certainty is dual to uncertainty -- the Heisenberg certainty/uncertainty principle. Syntropy (prediction) is dual to increasing entropy -- the 4th law of thermodynamics. Randomness (entropy) is dual to order (predictability) -- "Always two there are" -- Yoda.
@MrofficialC
2 жыл бұрын
And low entropy is easier to rig
@lani0
2 жыл бұрын
You win
Excellent explanation, very clear and concise! I have always pondered the significance of the log in cross-entropy loss function. The explanation (particularly: "products are small and volatile, sums are good") completely clears this up.
how does one make something so complicated into something so intuitive that others can finally see the picture. your explanation itself is an amazing feat.
Luis, you are such an incredibly gifted teacher and so meticulous in your explanations. Thank you for your hard work.
What a great explanation ! I wish I had a teacher like you Luis, everything wold be way easier ! Thanks a lot
I have been scared of delving into entropy in detail for so long because the first time I studied it, it wasn’t a good experience. All I want to say is THANK YOU!!!!!! I should have been supplementing the udacity ND lesson videos with these since the beginning.
Wow, thank you, man. I needed that information! There are many ways to teach the same stuff! That number of question stuff is great! It's good to have more than one way to measure something!
Thank you so much. This was the only video in youtube that clarified all my doubts regarding the topic of entropy.
This was one of the best explanations on entropy. Thanks
Thanks a lot Luis, just had an exam about this Wednesday and your video helped me a lot to understand the whole concept.
Great clarity. Have never got this idea about the Shannon Entropy. Thank you. Great work!
I needed this video to get me up to speed on entropy. Great job Luis!
2nd time I found this video and loved it both times. Much better description than the prof at the uni I am at!!!
The best explanation about Shannon entropy that I have ever heard. Thanks!
I'm studying Decision Tree (Machine Learning Algorithm) and it uses Entropy to efficiently build the tree. I finally understand the details. Thank you!!
Wow! Awesome, so books and encyclopedias and biographies of Shannon to understand what you just clearly explained! Thank You!
Great video! Now I understand what Claude Shannon discovered and how useful and essential maths are in Computer Science.
Gracias. Muito claro Senhor. I have been struggling to wrap my head around this and you just made it easy. Thank you.
Luis, You have a great way of explaining. At times , I like your videos more than even some highly rated professors
Great work. Compared to my textbook you explained it 100 times better, Thank you.
This video is helping to keep me floating in my Data Science course; thank you so much for your time!
Wow ..... I wish more people could teach like you this is so insightful
i wish i had this lecture during college examination.....still it's nice to finally understand the intuition behind the formulas i already knew.
@pixboi
5 жыл бұрын
Teaching should be like this, from practice to theory - no the other way around!
Easy and Great explanation! Thank you very much, Luis
Thanks for the relationship between knowledge and entropy, that was very helpful. Your explanation of statistics is also good! Though, I am only half way through the video at this point, I will finish it! Thanks
Excellent! Great explanation. Enjoyable video (except YT’s endless, annoying ads). Thank you for composing and posting.
That was highly intuitive, thank you, sir, I appreciate the effort behind this.
Very good explanation - hope to hear more of your videos
this is a really great explanation, thanks so much for sharing mate!
You are the best.Such a great explanation.Better than lots of text books.
amazing explanation!! very clear and articulate. Thank you!
It's very helpful for me to introduce the concept of entropy to students. Thank you for your clear presentation of entropy.
Very nice video. Insightful, inutuitive and very well explained. Thank you!
Lovely explanation...Superb
I have learned so much from your teaching. Thank you.
Thank you so much for explaining this concept!
Thank you very much for this beautiful and clear explanation!
Brilliant lecture! I learn so much with this explanation. Thanks from Brazil :)
Very clever explanation of mighty ENTROPY.
What a great explanation! And so i subscribed😊
Hola Luis, estupendo, espectacular, excelente!
I watched it straight through. Very good.
You explained it really very well. Thank you !
easy and thorough explanation, thank you !
Really, you have given us outstanding information.
Great and Superb. Thank you so much for the most clearer explanation.
Great video, simple and to the point.
Nice explanation. Thank you so much for this video.
Thanks..Got the intuition behind Entropy
Very well explained, thanks Luis
wow, another great and insightful presentation . really helps to build intuition
Luis, you really are a great communicator. Looking forward to your other explanations.
Thank you for the very good video. Easiest to understand so far.
Best explanation of entropy. Thanks.
十分谢谢! Thank you very much, Luis.
Superb step by step explanation
Beautiful explanation, thank you!
Thank you. Nice, concise explanation.
Thank you for excellent explanation of entropy concept first... Then reach to final equation step-by-step it is really good and simple way
Thank you very much. Very nice explanation.
Fascinating. Excellent Job
Lovely job Luis! Very very good!
Confession: I was a math kiddy; I know to use it but I often missed the deeper meaning and intuition. Your videos are turning me into a math hacker.
thanks a lot for such a beautiful explanation..!
great , It helped me alot . Thank you Luis
You are awesome!. I hope you will make more videos like this
thanks a lot, Luis. This was helpful.
It was a great explanation! Thank you.
Great explanation, greetings from Brazil!
Excellent presentation for an otherwise complex concept.
Best instructor there is! Thanks
Excelente explicación! Gracias por compartirla.
Mr. Luis Serrano III great job in Neural Network and Claude Shannon Entropy.
Very well explained, thank you
Nice explanations! Thank you.
hi Luis, nice to meet you, I am reading the book of Deep learning of Ian Godfellow, and I needed to view your video for understand the chapter, 3.13 information theory. thanks very much.
Thank you so much! Luis!
Amazing explanation, thanks sir!
Great, clear explanation! Thanks
wonderful video. Thanks!
Thanks for the great explanation.
best, as always ❤️ thank you Luis❤️
At 13:44 it's not 0.000488 but 0.00006103515 ! There is a computation error. The entropy is correct, 1.75.
@SerranoAcademy
4 жыл бұрын
Thank you for the correction! Yes, you're right.
Perfect explanation of this
Awesome explanation, you rock
Thank you very much for sharing. I found it very helpful.
Very well explained. Thank you.
Thank You for the great explanation.
Awesome video, very well explained.
Great Video! I really liked the intuitive approach. My professors was waaaay messier.
Super perfect explanation ! thanks
very lucid explanation - excellent, intuitive build-up to Shannon's theorem from scratch
awsome explanation on the formula sir
Very nicely done.
Awesome explanation!!
Best explanation I found so far
Fantastic work.
Thanks for this perfect explanation 👏👏👏👍
I love the explanation of the negative sign in the Entropy Equation many people wonder