Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

@thatguyadarsh

11 ай бұрын

Subscriiibed ! DOUBLE BAM!!! 😂😂

Still the best stat videos on KZread You have no idea how much you've helped me. You'll be in the acknowledgments of my diploma

@statquest

4 жыл бұрын

Wow, thanks!

@johnnyt5108

2 жыл бұрын

He'd probably like better to be in the acknowledgments of your checkbook then

@reflections86

Жыл бұрын

@@johnnyt5108 I am sure many people will do that by buying the reports and the book Josh has written.

@LauraMarieChua

Жыл бұрын

update after 2 years: did u include him on ur diploma?

The visualization really sells it.

@statquest

4 жыл бұрын

Thanks! :)

You are truly an angel. Your videos on Ridge, Lasso and Elastic Net really helps with my understanding. It's way better than the lectures in my university.

@statquest

3 жыл бұрын

Thanks!

Hello Josh, Ridge and Lasso clearly visualized :) I must say that if one thing that makes your videos clearly explained to curious minds like me is that the visual illustrations that you provide in your stat videos. Glad. Thank you very much for your efforts.

@statquest

4 жыл бұрын

Thank you very much! :)

Thanks, josh for this amazing video. I promise to support this channel once I land a job offer as a data scientist. This is the only video on youtube, that practically shows all the algo's.

@statquest

3 жыл бұрын

Thank you and Good luck!

I'm just gonna take a minute to appreciate the effort you put in your jokes to make the video more interesting. Its quite underrated.

@statquest

2 жыл бұрын

Thank you!

I was wondering why I missed out on this video while going through the ones on Ridge and Lasso Regression from Sept-Oct 2018. Then I noticed this is a video you put out only a few days ago. Awesome. Much gratitude from Malaysia. 🙇

@statquest

4 жыл бұрын

Thanks! :)

Best visuals ever! No matter how much I think I know about stats, I always learn something from your videos. Thanks.

@statquest

2 жыл бұрын

Thank so much! BAM! :)

Fantastic, Josh!! Thank you very very much. We all owe you a lot many thanks. "I" owe you a lot. 😊😊👍👍

@statquest

4 жыл бұрын

Awesome! Thanks! :)

Hi Josh, pse accept my heartfelt thanks to such a wonderful video. I guess your videos are an academy in itself. Just follow along your videos and BAM!! you are a master of Data Science and Machine Learning. 👏

@statquest

Күн бұрын

Wow, thank you!

Just found this channel today, great illustrations! Thanks for keeping the voice speed down, that makes me easy to follow!

@statquest

4 жыл бұрын

Awesome, thank you!

This explained everything i needed to know in 9 minutes. Absolute genius, thank you!

@statquest

3 жыл бұрын

Glad it was helpful!

I needed this information for my data science class and didn't expect such a well crafted and humorous video! You are doing great work sir!

@statquest

2 жыл бұрын

Wow, thank you!

These videos are so clear and fun, they helped me a lot with modeling and statistic in biology.

@statquest

3 жыл бұрын

Thank you! :)

You are a life saver! I have been trying to understand this for years now!! Thanks a ton!!!

@statquest

4 жыл бұрын

Bam! :)

dude is creating quality videos and replies to every comment! talk about dedication! thanks a lot

@statquest

5 ай бұрын

bam! :)

Great work Josh! Your songs get me every time.

@statquest

4 жыл бұрын

Bam! :)

this channel is saving my ass when it comes to applied ml class. so frustrating when a dude who has been researching Lasso for 10 years just breaks out some Linear algebra derviation and then acts like your suppose to instantly understand that...... thanks for taking the time to come up with an exhbition that makes sense.

@statquest

3 жыл бұрын

Thanks!

I've never been good with this kind of math/statistics because when I encounter the book formulas I tend to forget or not understand the symbols. Your videos make it possible to go beyond the notation and to learn the idea behind these concepts to apply them in machine learning. Thank you !

@statquest

2 жыл бұрын

Bam! :)

Can we start a petition to change the lasso and ridge names to absolute value penalty and squared penalty pwease?

@statquest

3 жыл бұрын

That would be awesome! :)

@JoaoVitorBRgomes

3 жыл бұрын

@@statquest I am listening to u on spotify

@statquest

3 жыл бұрын

@@JoaoVitorBRgomes Bam!

incredible videos, been watching all of your videos during quarantine for my future job interview. Still waiting for the time series tho. Thanks sir

@statquest

4 жыл бұрын

Thanks!

Many people on the Internet explain regularization of regression using polynomial features in the data that ridge and lasso are allegedly used to reduce the curvature of the line, but in fact in this case you just need to find the right degree of the polynomial. You are one of the few who have shown the real essence of regularization in linear regression and the bottom line is that we simply fine the model by exchanging the bias for a lower variance through slope changes. By the way, real overfitting in regression can be well observed in data with a large number of features, some of which strongly correlate with each other, as well as a relatively small number of samples, and in this case that L1/L2/Lasso will be useful. Thank you so much for a very good explanation.

@statquest

8 ай бұрын

Thanks!

Your are a super professor and I'll give you a infinity BAM !!!!!!!!!!!. I really like the way your repeat the earlier discussed topic to refresh the student memory and that really helpful and you have a-lot of patience. Once again you proved that a picture is worth a thousand words.

@statquest

3 жыл бұрын

Thank you very much! :)

You are really doing great great job. This channel is the best way to learn a lot, right and important things in a short time.

@statquest

3 жыл бұрын

Thank you very much! And thank you for your support!! BAM! :)

Thank you, regularization seris videos from 2018 to 2020 are so helpful.😀

@statquest

Жыл бұрын

Thanks!

Thanks a lot for this wonderful lesson...loved it ..seeing how the function behaves with different parameters makes it etched in the memory

@statquest

2 жыл бұрын

Glad it helped!

Thank you very much for this video! It helped me visually understand how Lasso regression can remove some predictors from the final model!

@statquest

3 жыл бұрын

Glad it was helpful!

You simply amaze me with each of your videos. The best part is the way you explain stuff is so original and simple. Will really love if you could also pen down a book on AI/ML. Would be a bestseller i reckon for sure. Keep up the good work and enlightening us :)

@statquest

4 жыл бұрын

Wow, thank you!

@rainymornings

Жыл бұрын

This aged very well (he has a book now lol)

You are awesome Josh. This one always bothered me why L1 would make coefficients to 0 and not L2 and you explained it so simply.

@statquest

3 жыл бұрын

Thank you! :)

Great videos, thank you very much!!!

@statquest

3 жыл бұрын

Glad you like them!

Very well explained this one video cleared all my doubts along with practical calculations and visualization. Kudos for the great job.

@statquest

4 жыл бұрын

Thanks! :)

This is amazing - thanks for this

@statquest

5 ай бұрын

Thanks!

"Unfortunately, no one asked me." I almost fart loud in a library.

@statquest

8 ай бұрын

:)

Dude you succeed at helping me and at making that thing funny as I'm struggling with my ML homework, thank you so much

@statquest

6 ай бұрын

Glad I could help!

The best. Definitely gonna come back and donate once I land a job.

@statquest

8 ай бұрын

Wow! Thank you!

Thanks for taking out time and explaining ML concepts in an amazing manner with clear visualizations. Great work.

@statquest

Жыл бұрын

WOW! Thank you so much for supporting StatQuest! TRIPLE BAM!!! :)

@heteromodal

Жыл бұрын

@@statquest Hey Josh! What's your preferred way of being supported? Would Paypal be better than Patreon?

@statquest

Жыл бұрын

@@heteromodal It's really up to you - whatever is more convenient and whether or not you want to be a long time supporter or not.

@heteromodal

Жыл бұрын

@@statquest I meant assuming i make a fixed sum donation - would you see more of it through PP or Patreon :)

@statquest

Жыл бұрын

@@heteromodal If it's a one-time donation, than PayPal is probably the best.

I just became the 104th patron of the channel!

@statquest

4 жыл бұрын

TRIPLE BAM!!! Thank you very much!!! :)

Great videos. Very helpful. Thanks !

@statquest

3 жыл бұрын

Glad you like them!

Thanks a lot for thes awesome videos, you deserver milllion followers, and a lot of credits :) I just love these and they are KISS. so simple and understandable. I owe you a lot of thanks and credits :D

@statquest

3 жыл бұрын

Thank you so much 😀!

Great, many thanks, very understandable and clear. it gave me a good intuiton of how lasso regression shrinks some varables to zero.

@statquest

Жыл бұрын

Glad it was helpful!

Very well done as usual.

@statquest

4 жыл бұрын

Thank you very much! :)

Dude you're killing it!

@statquest

4 жыл бұрын

Thank you! :)

Great master, thanks for your great effort

@statquest

3 жыл бұрын

Thank you!

Just an incredible explanation!

@statquest

2 жыл бұрын

Thank you!

Great explanation!

@statquest

Жыл бұрын

Thanks!

Ridge regression! Good topic to cover as always!

@statquest

4 жыл бұрын

Thanks! :)

Thank you so much Blessing from Spain/Morocco

@statquest

6 ай бұрын

Thanks!

Your videos are super intuitive.. thanks alot sir

@statquest

4 жыл бұрын

Thanks and welcome!

The explanation can't be any better than this....!

@statquest

Жыл бұрын

bam! :)

This is the perfect explanation I am searching for why L1 can be used for feature importance!!!

@statquest

2 жыл бұрын

bam! :)

Thank you for helping us to understand statistics! May I request for a video on Dirichlet regression?

Really enjoying these videos, Josh. Please keep 'em coming. Although I understand the distinction between correlation and interaction, I'd be interested to see how you might explain it in your inimitable fashion.

@statquest

4 жыл бұрын

I'll put that on the to-do list.

@gavinaustin4474

4 жыл бұрын

@@statquest I'm pushing my luck here, but one more item, if I may: the difference between PCA and factor analysis. Often, these are distinguished in general terms (e.g., they are concerned with the total variance vs the shared variance, respectively), but I think that the best way to distinguish them would be to apply both methods to the same data set. I would be most interested in seeing that done.

Thank you very much!

@statquest

2 жыл бұрын

bam!

That's a great video, Josh! 6:10 they should definitely have asked you 😂

@statquest

Жыл бұрын

BAM! :)

God of explanation !!! 🙏🏻🙏🏻🙏🏻 Awesome stuff 🙂🙂

@statquest

4 жыл бұрын

Thank you! 🙂

Excellent Explanations 👍👍👍 Great work 👍👍👍

@statquest

Жыл бұрын

Thank you!

this video.....you are my savior ❤️❤️❤️

@statquest

2 жыл бұрын

bam!

Thank you Josh

@statquest

3 жыл бұрын

Any time!

Fortunately, I asked you :) I agree squared and absolute penalty are better word choices for these regularization methods. Thanks again for making my ML at Scale a tad bit easier.

@statquest

3 жыл бұрын

BAM! Thank you very much! :)

The best visualization I've ever seen

@statquest

4 жыл бұрын

Thank you! :)

Nice. Thank you.

@statquest

Жыл бұрын

:)

As a result when the slope becomes 0 for large lambda in lasso, then we can use lasso for feature selection. Nice Video Josh!!.

@statquest

3 жыл бұрын

Bam! :)

And thats the reason why lasso does a kind of feature selection and sets many weights to 0 compared to ridge regression. And now I know the reason behind it thanks a lot❤

@statquest

4 жыл бұрын

BAM! :)

Just too much awesome!

@statquest

3 жыл бұрын

Thank you! :)

your videos're very explanatory for studying this field...

@statquest

4 жыл бұрын

Glad you think so!

Amazing series on regularization (As usual) I just didn't quite understand why in the ridge regression the weights/parameters never ever reach zero, I didn't give it much thought but it didn't pop right at me like it usually does in your videos lol, but again great series!

@statquest

Ай бұрын

Thanks!

All your videos are great, but the regularization ones have been a fantastic help. Was wondering if you were planning any on selective inference from lasso models? That would complete the set for me haha

@statquest

4 жыл бұрын

Not yet!

THAT IS SOOOOOO GOOD MAN

@statquest

3 жыл бұрын

Thanks! :)

Just wanna say..U r my Guru..means Teacher..In data Science ...more love to u from India

@statquest

Жыл бұрын

Thank you! :)

Amazing video as always Josh! Just to be sure if I got it correctly, the plot between RSS error and slope represents a parabola in 2D. So when we do the same thing in 3D i.e. With 2 parameters, does it represent the same bowl shaped cost function that we try to minimise?

@statquest

3 жыл бұрын

Yes

I first like your videos then watch them!

@statquest

2 жыл бұрын

BAM!

Can you do a lecture on Kohonen Self Organising Maps?

"Unfortunately, no one asked me" 🤣🤣🤣

@statquest

3 жыл бұрын

:)

This guy is amazing.... BAM!!!

@statquest

Жыл бұрын

Thanks! :)

Clear and apt..

@statquest

4 жыл бұрын

Thanks! :)

Hey josh!! Can u plz make a video for K-modes algorithm for categorical variables(unsupervised learning) with an example..plz?

"I got ... calling a young StatQuest phone" 😁 (The Ladys might love your work fam.)

@statquest

4 жыл бұрын

Bam!

Really excellent video Josh. You consistently do a great job, and I appreciate it. Could you make a video showing the use of Ridge regression and especially Lasso regression in parameter selection? I had to do that once, and it is complicated. From your example it seems that using neither penalty gives you the best response. So, in what circumstances do you want to use the regression to improve your result? If you are using lasso regression to find the top 3 predictive parameters, how does this work? What are the dangers? How do you optimally use it? A complicated subject for sure! I'm sorry if this is covered in your videos on Lasso and Ridge regression individually, I am watching them next. I agree with your naming convention btw, squared and absolute-value penalty is MUCH more intuitive!

@statquest

4 жыл бұрын

Watch the other regularization videos first. I cover some of what you would like to know in about parameter selection in my video on Elastic-Net in R: kzread.info/dash/bejne/laihsNNwdsrIpqw.html

@omnesomnibus2845

4 жыл бұрын

@@statquest I will check out those videos, thanks. I actually did use elastic net regularization. The whole issue is complex (for somebody without a decent stats background) because the framework of how everything works isn't covered very well AND simply anywhere that I could find, without going down several pretty deep rabbit holes. Some of the parameter selections that I remember were suggested depended on the assumption that the parameters were independent, which was NOT the case in my situation. I'm still not sure what the best approach would have been.

@omnesomnibus2845

4 жыл бұрын

@@statquest As an additional note, I've always found that examples and exercises are even more important than theory, while theory is essential at times too. In many math classes concepts were laid out in formal and generalized glory, but I couldn't get the concept at all until I put hard numbers or examples to it. It's probably not the subject of your channel or in your interest, but I think some really hand-holding examples of using these concepts in some kaggle projects, or going through what some interesting papers did, would be a great way of bringing the theory and the real world together.

@statquest

4 жыл бұрын

@@omnesomnibus2845 I do webinars that focus on the applied side of all these concepts. So we can learn the theory, and then practice it with real data.

@omnesomnibus2845

4 жыл бұрын

@@statquest That's great!

Hi, thanks for this explanation, it really helped! In my previous work place almost everyone said that lasso could be used for feature selection and it was kind of given. Like no matter what the lambda value is but it solely depends on the lambda right? It may not remove any features at all? And increasing lambad value to the maximum isn't always most beneficial?

@statquest

Жыл бұрын

That's correct.

L2= weight penalisation (smooths out weight losss curve but and reduces overfitting , but higher lambda can kill model training) L1 = weight imputation (dragging it to zero, useful for learnable ignoring of variables, useful for high dimensional data at times) . I have used both of these earlier with similar mindset. Earlier even in Deep Learning i used a similar analogy to reason about what was happening. The visualisation really did helped, so just wanted to know is this simplistic way of viewing the behaviour makes sense ??? Or am I missing something ....

hey josh ur videos are great, ı m really appreciate for that but i have a question: at 0:46 when u choose the line with 0 slope, why did u choose its intercept at that value, bcs if u choose another intercept value u ll get totally different L1 and L2 graphics.ty

@statquest

2 жыл бұрын

I think the intercept is from the least squares optimal fit. I picked it because it does a good job illustrating the concepts without making the explanation more complicated than it needed to be. In theory every line will have a different intercept, but the concepts illustrated will not change.

Perfect!!!!

@statquest

4 жыл бұрын

Thanks! :)

Awesome! And I should mention actually: We are asking YOU!"

@statquest

3 жыл бұрын

Bam!

Thank you very much for the video!!! All u r Videos are really easy to understand... thanks alot.. could you please!!.... upload a video for SCAD (Smoothly Clipped Absolute Deviation Method) Regularization Method....

@statquest

2 жыл бұрын

I'll keep that in mind.

Genious!

@statquest

2 жыл бұрын

Thanks!

You should receive a Nobel Prize.

@statquest

3 жыл бұрын

BAM! :)

Hi Statquest. Thank you for the video. Just have a question. Let's say we have a categorical variable with 4 levels. After running the LASSO regression, if we see 3 dummy variables' beta (i.e. 1 level is in the base level) are 0 then do we drop that categorical variable? I know if the LASSO drops only 1 dummy variable, not 3, we are just merging that dummy variable with the base level. Thanks as always

@statquest

9 ай бұрын

To be honest, I don't know exactly what will happen in that situation. What you suggest, dropping the variable seems reasonable, but I'm not sure.

@shawnkim6287

9 ай бұрын

@@statquest thanks. If a numerical variable's coefficient is 0 then we are dropping that predictor in our lasso model?

@statquest

9 ай бұрын

@@shawnkim6287 yes

Josh, great video, I got a quick question: In your plotting, when lambda goes bigger (say lambda =10), (the sum of squared residual + penalty) is always overlapping or higher than the case when lambda = 0, then why do we want to use Ridge/Lasso even the sum is bigger than when lambda = 0.

@statquest

3 жыл бұрын

The individual videos on Ridge kzread.info/dash/bejne/g2xltLRsqa7UY5M.html and Lasso kzread.info/dash/bejne/gHuaktiohLDSk9Y.html regression show why we would want to use regularization. In short, Ridge and Lasso regression can improve predictions (and this can be observed with cross validation). In contrast, the purpose of this example is only to highlight the differences between Ridge and Lasso regression and show why Lasso can set parameters to 0 if lambda is large enough.

Thank you for your work as always. Its AWESOME. I just got some questions. Why is there a kink in the SSR curve for Lasso Regression ? Is it because we are adding lambda * |slope| which is a linear component ? And Does the curve for Ridge Regression stay parabola because we are adding lambda*slope^2 which is a parabola component ?

@statquest

3 жыл бұрын

I believe that is correct.

@sudeshnasen2731

2 жыл бұрын

Hi. Great video! I had the same query as to why we cannot see a similar kink in curve in the Ridge Regression CF vs Slope curve.

curious is it possible to learn the optimal y-intercept or the optimal bias term with regularization ? any insights? Like get a worse fit straight line with slope zero but cutting y axis at a different height?

@statquest

2 жыл бұрын

As we increase regularization, the y-axis intercept moves towards the average y-axis value, which is the optimal value if none of the parameters are useful.

Thank you for such a great video. I have a question though: What happens to the intercept term during regression? Do you also shrink it? Here you have it fixed and only vary the slope but in your older ridge regression video you changed both!

@statquest

3 жыл бұрын

The intercept is not affected by regularization. For details, see: kzread.info/dash/bejne/g2xltLRsqa7UY5M.html

Underrated

@statquest

3 жыл бұрын

Thanks!

Was about to start this topic. Thanks @Statquest Hellowww , I just have quick question. by what time we should expect your video on Neural networks? And I have a request, could you add your upcomming videos on your website in a separate sections. By looking at the topic and the date odr month it will be uploaded. It will be very helpfull for students to buy a subscription plan of your channel in order to get early access to your videos.

@statquest

4 жыл бұрын

You can get early access by becoming a channel member or signing up for my Patreon: www.patreon.com/statquest

please do: Wasserstein metric (earth movers distance)

Could be interesting to see the explaination in the case of a multidimensional problem with more than 2 d features, but very nice video!

@neillunavat

3 жыл бұрын

Be grateful we've got such a nice guy.

‘Unfortunately no one asked me ‘ 😀 . Unique content . Hats off !

@manjushang

3 жыл бұрын

Also it will be of great help if you explain the following points. 1. How the lasso regression excludes the useless variables. 2. How the ridge regression do a little better when most variables are useful Thanks, Manjusha

@statquest

3 жыл бұрын

Thanks!

I died laughing when you sang the intro.

@statquest

2 жыл бұрын

That's a good one! :)

Explain stats to a 10-year-old? Me: "You kid, Subscribe and drill through all the content of StatQuest with Josh Starmer"

@statquest

4 жыл бұрын

:)

Excellent. I have just one question. In case of L1 penalty, isn't the line with lambda equal 40 (or slope 0) giving a bad line? I mean with blue line, we were getting a better fit since it didn't completely ignore weight in predicting height and sum of residuals is smallest?

@statquest

2 жыл бұрын

What time point, minutes and seconds, are you asking about?

@usamahussain4461

2 жыл бұрын

@@statquest 7:16

@statquest

2 жыл бұрын

@@usamahussain4461 Yes. For both L1 and L2 you need to test different values for lambda, including setting it to 0, to find the optimal value.