You just simply explained a complex topic that I spent 3 hours on reading a textbook into 10-minute video. Your ability to condense and concisely explain these topics in your videos have been phenomenal. Great job!

@Mutual_Information

Жыл бұрын

Thank you! Glad this is working for you

Thanks for the intuition-nurturing graphics. Very helpful!

@Mutual_Information

3 жыл бұрын

Glad it was helpful!

Congrats on the launch of your channel and first video, DJ! This was awesome!

@Mutual_Information

3 жыл бұрын

Thanks Sumaya! More coming :)

Intuition is indeed what helps at least me to understand (not just short-term memory) a concept, great work, thank you!

@sskhdsk

Ай бұрын

Human transforms short-term memory to long-term memory by understanding and prediction.

Probably one of the most underrated inequalities... Shows up everywhere (mostly Machine-Learning these days, but I also encountered this in neutron transport and rendering of images)

You have no idea how often your explanations blow my mind. It is an "aha" moment every single time, a concept clicks so well! Please keep up this amazing work.

@Mutual_Information

Жыл бұрын

THank you and I will! I got something big in the works :)

@karanshah1698

Жыл бұрын

@@Mutual_Information Do you plan on doing one for EM derivation of GMMs?

@Mutual_Information

Жыл бұрын

@@karanshah1698 EM yes, GMMs, yes eventually. Using them together?? No I didn’t think of that, hm

You sure have a gift for teaching ! Plus, what a slick production . It takes a lot of hard work and skill to make something look as simple and obvious as you do. Awesome.

@Mutual_Information

Жыл бұрын

I appreciate that!

I was curious about Jensen’s inequality, having seen it in the context of EM. You did a great job of providing even more context and explaining the intuition. The animation makes it so easy to understand why it is true. Simply outstanding. This is hands-down the best video I’ve seen on the subject. Thank you! Just subscribed.

@Mutual_Information

Жыл бұрын

Thanks - great to have you! This was my first vid. I've gotten a lot of useful feedback since, but glad this one still lands

Then for concave function, I expect 'greater than equal to', instead if less than equal to.

Great explanation. Instead ofaverageg, its better to think of weighted average; this will easily convey the idea of the formal definition of convex function : )

Wow. The way you simplified the concept. I was amazed.😍

Great explanation, I am at the process of figuring out the cross-entropy and you video helped me with Jensen inequality concept! Keep it up!

@Mutual_Information

3 жыл бұрын

Thank you! Happy to help

Great visualisation, really good job! Thank you very much!

As a tired econometrics student with a dull lecturer, this helped a bunch, thanks

Very interesting. Keep ‘em coming DJ!

@Mutual_Information

3 жыл бұрын

First comment :) will do!

nice quality and explanation, really helped me out

Nicely and Intuitively explained! Thanks

Great visualization, thank you very much for your effort to break it down so well!!! :)

Thanks for such great visuals

Great video! Thanks for sharing your knowledge!

Thank you very much for this wonderful presentation. A lot of effort must have gone into getting an animation that feels intuitive.

Great intuition, great visualizations! Mathematicians will also say that you need to assume that X is integrable in order for Jensen's inequality to hold. Jensen also has far reaching consequences in theoretical probability, and even analysis in general. Can't wait for more!

@Mutual_Information

3 жыл бұрын

This means a lot getting your comment here. Much appreciated! And yes! There are unfortunately rigor qualifications that I omit to keep the vid light. In the case of integrability, I hadn’t thought of that, so thanks for pointing it out :)

Great Video ...Thanks for the efforts you put in these Videos ..🙂

This was an awesome explanation. Thank you! Out of curiosity, how did you make the animation? By the way, that was also really well made!

@Mutual_Information

2 жыл бұрын

Hey thanks and to answer your question, I stick together a bunch of graphs made in Altair using a personal library. Altair is very nice plotting library

@asdfasdfuhf

Жыл бұрын

Looks like he is using manim made by 3blue1brown

Very clear explanation. Keep up the good work

You basically took an esoteric formula and explained it in a stupid-people friendly way. Thank you

@jacoboribilik3253

10 ай бұрын

There's no need to put yourself down in such a way. You can be thankful for the content this guy is putting out on YT by liking, subscribing and hitting the share buttom...stop dragging yourself over the coals.

Great video, hopefully there is a followup videos on how Jensen's Inequality becomes the important part of EM, KLDiv and so on.

@Mutual_Information

3 жыл бұрын

Yes! The EM algorithm will be covered, but later this year. If you're curious immediately, I linked to some sources in the description where Jensen's Inequality is used. In Cover's book, there is a section "Jensen's Inequality and Its Consequences" which show how foundational it is for Information Theory.

Amazing video! Subscribed.

Perfect presentation, thank you!!

Awesome video! Really easy to follow

Your channel looks great, thanks!

Nice animation making things a lot more intuitive, thanks.

Amazing explanation!

Outstanding explanation!

Thanks...you've explained it clearly

You are awesome. I have been binge watching your videos.🖖

@Mutual_Information

Жыл бұрын

Excellent, thank you!

Huh.. it clicked in like 3 seconds after seeing the comparison with line. And why it's true is so obvious now. Amazing. Thanks.

Brilliant explanation!

incredibly clear and helpful! thx a lot

very clear explanation with high energy .. I like it

@Mutual_Information

7 ай бұрын

lol old video will lots of energy.. I've chilled out a bit since, but thank you

Great explanation!

Excelent explanation indeed!

This was awesome! Could you make a video about where this is applied? you talked about how it effects ML but could you show an example? Thank you! btw the algorithm showed me this video so hopefully ur on the rise! Honored to be this early

@Mutual_Information

3 жыл бұрын

Yea I’ll do a video on the EM algorithm, where this shows up. Also, variational inference, eventually. And I’m happy to hear that! I hope you’re right but we’ll see.

You deserve more likes!

Very good explanation without keeping it complicated, thanks a lot/

@Mutual_Information

2 жыл бұрын

Course man - glad you liked it

Thanks. Very interesting. I listend to a talk from Nassim Taleb where he talked about the Jensen’s Inequality.

@Mutual_Information

3 жыл бұрын

Yea it shows up a lot. Glad you enjoyed it

You are awesome...this explanation is so cool.

@Mutual_Information

3 жыл бұрын

Glad you like it!

This is relevant to extractor theory used in cryptography (usually as min-entropy, not the Shannon entropy you assume here) - how does your function change the entropy per bit of the input data?

Nice visualization!

GREAT VISUALISATION TY

This is so good. Thankyou.😊

@Mutual_Information

Жыл бұрын

And thank you too Satish!

Awesome video!

@Mutual_Information

3 жыл бұрын

Thank you! More coming - one every 3 weeks.

Just found your channel through your comment on 3b1b's video, very nice explanation. Btw, is the opposite inequality true for concave functions?

@Mutual_Information

2 жыл бұрын

Glad you’re here! His channel is a huge inspiration. And yep, concave functions you get the opposite. The neg of a convex I’d a concave function and that deserves the inequality.

Liked and commented. Thank you and more please!

@Mutual_Information

2 жыл бұрын

I got something good cookin :)

Love the videos!

@Mutual_Information

3 жыл бұрын

Thanks!

Great vid bruv

great channel

Amazing! Thank you very much!

@Mutual_Information

Жыл бұрын

Thank you very much!

most OP explanation of all time. My intuition so far had came from showing it w the definition of convexity, This was awesome, relating it to N sampling was the key. Learning Online Convex Opt, any tips :p prof is planning 50% avg midterm

What software did you use for this cool moving graph thingy??

@Mutual_Information

2 жыл бұрын

Thanks for the love! Answered you on Twitter :)

Very Very nice!

Gem of a video...

Thanks !

Thank You boss

If you have a concave function, does the inequality sign just get flipped? Thank you very much for the content!

@Mutual_Information

10 ай бұрын

Yes it does!

Awesome! So ... given a random variable X I can use Jensen's inequality to estimate the local curvature of a function 'f' ?

So what would happen if the space of points below(!) the function is convex? Will we get the different inequality?

@Mutual_Information

3 жыл бұрын

Yep, then that would be a concave function and the inequality would be reversed. A pretty common example of that is the log(x) function.

great vid! but it would be even better if you can show the formal proof and connect it with the visualization u showed

Is it possible you could made a video on Variational Inference and the intuition on the loss function?

Cool inequality! If we take f(x) = x² then this inequality tells us that for any real a, b: a² + 2ab + b²

Thank you for this..

@Mutual_Information

Жыл бұрын

For sure - If you'd like, you can do me a solid and anyone into ML/stats about the channel :)

Can you upload some content about estimation maximization with mixed poisson?

@Mutual_Information

Жыл бұрын

Hm, sorry but that's unlikely. It's just too specific. The topics I've picked are already fairly niche. If I go into a very specific subtopic, it'll appeal to very few folks (unless there is something particularly fascinating about it)

3:19 in and I get it. Thanks!

Subscribed

I never understood why this inequality was true or what it really meant the first time I saw it, and my engineering prof said don't worry about the intuition just know how to use it (bleh), but KZread somehow knew to suggest this to me months later! Thanks for the clear explanation

@Mutual_Information

3 жыл бұрын

Glad you enjoyed it!

My friend you are amazing like really i feel bad how much time i wast in trying underatand it when you explain it amaizing in 5 minute Can you also make video on vae and variational inference elbo and all that? Please it is really topic which hard for a lot of people and look like it exactly in your domain Please

@Mutual_Information

2 жыл бұрын

Thanks! And I do have plans to make a video on variational inference but it may take awhile. There are a few videos in front of it. But it’s coming!

Heh. 4:55 "what mathematicians will ask, and engineers probably won't, is 'why?'." That definitely matches my own experience.

Can't thankyou enough!

Holyshit, this is a great video!

@Mutual_Information

3 жыл бұрын

Thank you - it means a lot !

Hey, nice video. I though we would see the f((a+b)/2)

@Mutual_Information

3 жыл бұрын

Glad you enjoyed it!

I wish I had friends like you! What's on the bookshelf? =) The intuition follows immediately from grokking the idea of convex functions! Some of the coolest tricks in mathematics come from manipulation of inequalities. There's a brilliant little maths book named "The Cauchy-Schwarz Master Class" which I recommend for anyone wanting to master these dark arcana. Sometimes, like in this case, an animation is just unbeatable, however!

@Mutual_Information

2 жыл бұрын

Thank you! I’ve heard that book recommended a few times but have never checked it out. I’ll order it!

I don't like the chosen function for the visualization because it is everywhere increasing. From this video i m not convinced that this reasoning would be valid for say a parabola segment.

The visual explanation is excellent. However, it might seem that it does not hold if one picks a convex function that is below the straight line. For completeness, it is worth remarking that the equality holds for any straight line. Because of this, for all convex functions f(x) there is always a straight line ax+b for which f(x)>=ax+b for all x. (Namely, the line you pick has to be the tangent of the convex function at E[X]).

@erikysilvagomes5496

Жыл бұрын

My question was not answered by the author, maybe you can help me. My doubt is precisely about the equality holding for any straight line... why it is true? I can understand it holds for linear functions of type f(x) = c.x, in the sense of addition and multiplication conservation -> hence f(E(x)) = E(f(x)). Functions of the type f(x) = ax+b are straight lines, but not linear funcions in these sense, so we cannot prove that f(E(x))=E(f(x)). Try to use f(x) ~ exp(k) and the transformation Y = ax + b, you'll see that f(E(x))=a/k+b and E(f(x))=exp(kb/a)*a/k, which do not hold for any straight line, but just for b=0.

@rocamonde

Жыл бұрын

I’m not sure what you’re saying. The equality is satisfied when f is an affine transformation. This is trivial because E is a linear operator: E(f(X)) = E(aX+b) = a E(X)+b=f(E(X)) where the first step applies the definition that f is affine (a straight line), the second, that E is linear, and the third using again the definition of f being affine. In your example you’re mixing up letting f be an exponential or letting it be an affine transformation, so you get something weird. You can’t apply both transformations at the same time, that is not something that Jensen inequality talks about in any way. If the function is convex, you get an inequality. If the function is strictly convex (unless X is constant a.s.) you get strict inequality. And in the opposite edge case that the function is affine (which is also convex), you get strict equality.

@erikysilvagomes5496

Жыл бұрын

@@rocamonde Of course you are correct, in my example I made a mistake calculating E[f(X)], for that reason the equality does not hold. My approach was to see the theorem from the affine functions properties, but in fact is much more simple to see it from the expectation linearity. Thank you, your explanation was clear!

Thanks!

@Mutual_Information

Жыл бұрын

Thank you! 3rd donation ever :)