NOTE 1: The StatQuest PCA Study Guide is available! app.gumroad.com/statquest NOTE 2: A lot of people ask about how, in 3-D, the 3rd PC can be perpendicular to both PC1 and PC2. Regardless of the number of dimensions, all principal components are perpendicular to each other. If that sounds insane, consider a 2-d graph, the x and y axes are perpendicular to each other. Now consider a 3-d graph, the x, y and z axes are all perpendicular to each other. Now consider a 4-d graph..... etc. NOTE 3: A lot of people ask about the covariance matrix. There are two ways to do PCA: 1) The old way, which applies eigen-decomposition to the covariance matrix and 2) The new way, which applies singular value decomposition to the raw data. This video describes the new way, which is preferred because, from a computational stand point, it is more stable. NOTE 4: A lot of people ask how fitting this line is different from Linear Regression. In Linear Regression we are trying to maintain a relationship between a value on the x-axis, and the value it would predict on the y-axis. In other words, the x-axis is used to predict values on the y-axis. This is why we use the vertical distance to measure error - because that tells us how far off our prediction is for the true value. In PCA, no such relationship exists, so we minimize the perpendicular distances between the data and the line. NOTE 5: A lot of people wonder why we divide the sums of the squares by n-1 instead of n. To be honest, in this context, you can probably use 'n' or 'n-1'. 'n-1' is traditionally used because it prevents us from underestimating the variance - in other words, it's related to how statistics are calculated. If you want to learn more, see: kzread.info/dash/bejne/qJ2fzcuwmrTXlrg.html kzread.info/dash/bejne/ha6OmKmpk8nVgbw.html and kzread.info/dash/bejne/pXyGpMlvcsuxmqw.html (the last video specifically addresses the 'n' vs 'n-1' thing, but the first two give background that you need to understand first). Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

@statquest

4 жыл бұрын

@@clydemccoy1768 If you can tell me how to draw a 5-dimensional graph on a 2-dimensional plane (the computer screen), then we have a deal.

@statquest

4 жыл бұрын

@@clydemccoy1768 There are two ways to do PCA. The original way was to apply eigendecomposition of the covariance matrix. However, that method is not numerically stable, so no one uses it anymore (even though it is still taught as if it is used). The preferred and more modern approach is to use Singular Value Decomposition (SVD), and this is what I cover in this video. At least, I cover the concepts behind how it works. However, I may dive into the math behind singular value decomposition one day. It's a long term goal of mine, but it won't happen for a while.

@KarthikNagadevara7791

4 жыл бұрын

@@statquest Thank you Josh :)

@ersincine3248

Жыл бұрын

2:40 Actually 4-D data can be plotted easily. 4th dimension can be specified by color. Shades of gray would be sufficient. For example a very dark gray (i.e. low "intensity") indicates a small number.

@kacodemonio

Жыл бұрын

Could you please clarify me the following: some sources mention that in addition to centering the data (subtraction of the corresponding mean), it must be also standarized, i.e., divided by their corresponding standard deviation. Why is the data not being normalized here?

For someone who is trying to be a data scientist, this channel is the best thing on the internet. You're better than any other teacher that I've ever had. THANK YOU.

@statquest

3 жыл бұрын

Wow! Thank you very much!

@cindywang8852

3 жыл бұрын

Can't agree more! This is my second time viewing this PCA video!

@grahamjoss4643

3 жыл бұрын

@@statquest Safa is right !

@joshuasheppard7433

3 жыл бұрын

@@statquest I agree entirely, very well explained.

@ZenithClown

3 жыл бұрын

Exactly, I still sometimes come back to this channel to refresh my memories!

6 years into my phd and I finally understand how a PCA plot actually works. Thank you!

@statquest

3 жыл бұрын

BAM! :)

@anjalikrishna203

2 жыл бұрын

Same!!!!!!

@adipurnomo5683

2 жыл бұрын

What research topic / thesis did you do sir @Dee T?

@stallman3738

Жыл бұрын

Where did you study? :|

@wumichael487

Жыл бұрын

little bam

Dr. Starmer - I really really wish I had you as my stats teacher during my student days. I can't put in words how much I appreciate your videos and how to go about explaining core concepts. Thank you very much!

@statquest

Жыл бұрын

Wow, thank you very much!!! I really appreciate your support! It means a lot to me that you care enough to contribute.

I can say without a shadow of doubt that you are one of the world's best teachers. Mr. Starmer I can never thank you enough.

@statquest

9 ай бұрын

Hooray! Thank you very much!!! :)

I am a data scientist and have performed PCA using advanced statistical softwares . I have even taken company sponsored expensive MVA courses. THIS is the best explanation of PCA I have seen and cleared my fundamental doubts and missing links. THANK YOU.

@statquest

4 жыл бұрын

Hooray! I'm glad the video was helpful. :)

@AD-dy1hi

2 жыл бұрын

Can you please help me with the names of best software for PCA analysis!?

@SHARMAKUMAR-zl6ku

Жыл бұрын

Hi kush i request you to guide me to become a data scientist like you

This is honestly the best explanation on PCA. connects lot of loose strings of ideas. thanks !!

@statquest

5 жыл бұрын

I'm so glad to hear that you like this video! :)

@sahej97

5 жыл бұрын

Then you haven't watched any other PCA videos

@DragonDaron

4 жыл бұрын

I agree 👍

@AnanyaPandeyIndia

3 жыл бұрын

@@sahej97 Not everyone is a born genius like you some others need detailed explanation. You know Albert Einstein said if you cannot explain to a 6 year old you have not understood it yourself.

@fisherh9111

3 жыл бұрын

Josh is brilliant.

This is the most helpful video on PCA I have ever seen. I'm a neuro PhD student and I was struggling to understand this concept for a year. This video is a life saver!

@statquest

Жыл бұрын

Thanks!

I had to pause and let you know, that this is gold! The way you simply describe terrifying names, it just makes it look so easy. Thank you so much

@statquest

8 ай бұрын

Glad you liked it!

uni student day 14 of self-quarantine: the first thing to do in the morning is to watch StatQuest. Thank you Josh. Your videos help me surviving thru my uni in the time of covid19

@statquest

4 жыл бұрын

Awesome!!!! I'm so glad to hear my videos are helpful and good luck with your courses.

I'm studying for a final and this is way better explained than the entire semester.

@statquest

4 жыл бұрын

Awesome! Good luck on your final. :)

@user-xn4yu5rn9q

4 жыл бұрын

Hi, what class is this you are preparing? Just curious

@ireliatt2190

4 жыл бұрын

@@user-xn4yu5rn9qFor me it's Multivariate Data Analysis. Honestly, I got totally lost when just reading the lecture notes.

@edchavez9960

3 жыл бұрын

@@user-xn4yu5rn9q Exploratory Data Analysis quite similar to @irelia tt

@edchavez9960

3 жыл бұрын

@@ireliatt2190 ha! mine was Exploratory Data Analysis, which included Supervised/Unsupervised learning and multivariate data analysis

I love how you explained "linear combination is just basically this... no big deal" when my lecturer makes it such a BIG DEAL! Thank you!!!

@statquest

3 жыл бұрын

Hooray! I'm glad the video was helpful. :)

Buddy you are an awesome guy. I wish one day I could give you a big hug of appreciation, but my honest thank you much has to be sufficient. I'd bet that there are people out there that would not be alive today without you. With your amount of views, there is probably at least one person who succeeded an important test that he would not have without you which may ended in a bad life. You're doing pure good with those videos.

@statquest

Жыл бұрын

Thank you very much! :)

Thanks dude. You are a godsend

@AhmedIsam

5 жыл бұрын

@19:41 should be triple bam

@pyarepiyush

5 жыл бұрын

BAM!!!!

@AndreDiasRJ

4 жыл бұрын

Double BAM!

@hemanthkumarb9444

3 жыл бұрын

Bam is not sufficient!

I'm in my first semester of grad school for stats.. and you are single-handedly going to save me! I already knew of all of the concepts of eigenvalues/eigenvectors, loadings, etc. but you summarized the 60 pages of theorems/proofs my prof had us read, and helped my intuition immensely. Please never stop making videos!

@statquest

3 жыл бұрын

Thank you very much and good luck with graduate school. :)

You are a real godsend. Teaching such advanced concepts in such a simple manner and that too in just 20 mins is exceptional

@statquest

3 жыл бұрын

Thank you very much! :)

I have forgotten how much I love maths. You are such a good teacher.❤❤. Instead of wasting time on shorts or tiktok, I am gonna watch this, learn something and improve my life instead. Thanks.

@statquest

10 ай бұрын

bam!

I just took some other online course for hours only to struggle... this video helped me understand the concept, which that course couldn't! Even in way shorter time! And even with the minimum animation! Always grateful

@statquest

5 жыл бұрын

Thank you very much!!! I'm glad to hear that the video was so effective, even though it's not very fancy. :)

Your work, with 3Blue1Brown, and other online material like Dr.Gilbert Strang, has given me a whole new perspective about Maths, learning and machine learning.

@statquest

4 жыл бұрын

Awesome! Those guys are my heros!

I lost it at "Little BAM!!!"

@statquest

3 жыл бұрын

:)

@cate9541

2 ай бұрын

Same😂

Please never stop making videos. This is the only explanation of PCA I've seen that makes intuitive sense.

@statquest

2 жыл бұрын

Thank you!

I honestly think you deserve a noble prize or something. You're amazing. Like, just amazing!

@statquest

4 жыл бұрын

Thank you! :)

I've been watching quite a lot of your videos for the past few days to study for my finals and I'm so glad your videos exist! I'd be very desperate without them. Thank you so much for the amazing work! You have a natural talent of expressing complicated ideas so simply!

@statquest

4 жыл бұрын

Good luck with your finals and let me know how they go. :)

This is such a complex topic and with the visualization you made it so simple, you really are magical in explaining the concepts, a million thanks for this Josh

@statquest

Жыл бұрын

Thank you!

Congratulations! This is the best explanation of PCA without overcomplicating things with matrixes. In my opinion, everyone who explain PCA should start from an explanation like this... Thank you for the amazing video!

@statquest

2 жыл бұрын

Wow, thanks!

I have seen no book explaining this topic better than you. Your skills and efforts are invaluable!

@statquest

3 жыл бұрын

Wow, thank you!

*with sad voice* "Little bam..." Awesome video. Thanks!

@cyclone975

4 жыл бұрын

Little bam got me too 🤣

As a non-English speaker, I just like, I have no words, I don't no how to tell you how much I appreciate the way you put almost your "lyrics" or whatever you said on the videos that help me a lot. Many thanks to you, Josh!

@statquest

2 жыл бұрын

Thank you! :)

Hi Josh , I should say this that you make even the more complicated logic so simple for us to understand . For people like us who are in learning phase , that means a lot . Thank you very much .

@statquest

3 жыл бұрын

Thank you! :)

This is simply amazing! Josh, you communicated this idea so incredibly clearly and simply. As a practitioner in the stats/data science world I'm so impressed by how you were able to describe PCA so intuitively and without using the word "orthogonal." I've done the math behind PCA and I actually feel like I have a better understanding of it by watching your video. Thanks for demystifying stats for us all.

@statquest

4 жыл бұрын

Thank you so much! I'm glad you picked up on my omission of the word "orthogonal". :)

Thank you very much! Opening my textbook, I was frightened by the Zs, phi, and "principal component directions" everywhere but as always your videos enable me to see the logic and fearlessly read the jargon :P

@statquest

4 жыл бұрын

Awesome! :)

Very impressive! I searched a lot of websites and you are the only one that make the conceptions clear. Your graphs and explanations are vivid and easy to understand.

@statquest

2 жыл бұрын

Thank you very much! :)

simply amazing. I just hope u never stop teaching us those 'complex' topics in such friendly and clear way.

@statquest

2 жыл бұрын

That's the plan!

I took a whole seminar on this thing complete with a mini project and I didn't understand a single thing. Then comes along this little video and makes it all crystal clear in just 20 min! Hats off to you sir.

@statquest

Жыл бұрын

Hooray! :)

"Mathematicians call this recipe a linear combination". I would have called it a Gene tonic!

@statquest

3 жыл бұрын

This might be the best comment ever. BAM! :)

I've been looking for a clear explaination for eigenvector when i accidentally found it in a PCA video by StatQuest. You sir is a savior.

@statquest

3 жыл бұрын

Bam! :)

You really made me feel the mathematics in PCA....... earlier I can solve the math in PCA but now I know what's the purpose of each step in finding PCA components. I'm a B.tech student of final year. Thank you ❤️

@statquest

2 жыл бұрын

BAM! :)

This man is helping me get through my diploma in Business Analytics. I have no background of stats or programming whatsoever. Could you imagine how intimidating the situation is for me. man thank you!

@statquest

4 жыл бұрын

Hooray! Good luck with your degree. :)

This is what a good teacher can do to you ! Now I can explain PCA even when I am woken up from sleep ! Thank you Josh ! You have amazing talent at teaching !

@statquest

4 жыл бұрын

Thank you very much! :)

Great job. I've rewatched this countless times today until it became as easy to explain to others as it was to understand ! You're truly a gifted teacher ! Keep up the great work ! Cheers from France !

@statquest

4 ай бұрын

Merci!

When I search for PCA on youtube and I saw that Statquest has a video for it, I was sure that I will learn PCA without any pain. Thank you!!

@statquest

3 жыл бұрын

Bam!

This is the easiest way I understand PCA. I've been working with PCA algorithms for 2 years and two of my university professors provide lesson on PCA but couldn't make it clear. But, watching this 20 minutes video I understand in depth. Such a good lesson. Thank you so much!! ☺️ 😊

@statquest

4 жыл бұрын

Awesome!!! I'm glad the video was helpful. :)

This is the best PCA video ever, finally understand how PCs come out, Josh u r genius

@statquest

4 жыл бұрын

Wow, thanks!

JUST WOW!! Never I had thought that Stats could be this efficiently & easily understandable!! I feel immensely lucky that the content of this quality is freely available on the internet!! Thanx a lotttt for this magnificent effort from the whole team of StatQuest😇

@statquest

10 ай бұрын

Glad it helped!

@silentassassin4929

7 ай бұрын

How ur calculating the parts that u said like 4 parts of gene 1 and 1 part of gene 2 please respond cannot understand

I've been trying to understand PCA for a while and this explanation made it look so simple, thank you!

@statquest

3 жыл бұрын

Glad it helped!

This is without question the best explanation for PCA I have ever seen. Much clearer than my PhD multivariate statistics course. Thanks so much!

@statquest

4 жыл бұрын

Awesome! :)

No words to tell how this video helped me! Thanks a lot, Josh! Your way to explain is extremely simple and makes anything easy for us.

@statquest

3 жыл бұрын

Thank you very much! :)

You boil all of this down into very understandable bite-sized steps. Amazing job! thanks for your work in explaining this.

@statquest

7 ай бұрын

Thank you!

This youtube channel has created a lot of interest in me for Machine learning concepts. I totally appreciate the efforts by Josh Starmer. Thanks Josh for the excellent explanations. By the far, the best tutorial videos on data science in the internet.. Free of cost !!!

@statquest

2 жыл бұрын

Thank you very much!!! :)

you took 10 minutes to blow my mind with your amazing explanation. Thanks!

Wow man thank you! Just had a moment of clarity, teachers like you are rare!

@statquest

4 жыл бұрын

Hooray!!! I'm glad the video was helpful. :)

the best PCA explanation that exists. very clearly eplained and without the technical jargon that over complicates supposedly uncomplicated explanations! thanks so much!!!

@statquest

2 жыл бұрын

Wow, thanks!

Hello Josh, I wanted to thank you. You are one of those great teachers who makes learning fun and easy. I’d prefer watching your videos again over a good movie. Thank you so much for all of your videos.

@statquest

3 жыл бұрын

Wow! Thank you! :)

You saved my future!!! Oh my god, this is amazing! You are a talent who can transfer complex to simple.

@statquest

4 жыл бұрын

Awesome! :)

As a current high-ranked uni student may say this channel is a lifesaver for people who cannot understand something without guided decomposition of relatively more complex algorithm than OLS. Thank you Josh.

@statquest

4 жыл бұрын

Thanks!

It is unbelievable how Dr. Starmer makes such complex concepts so easily understandable!!! Thanks a lot!

@statquest

2 ай бұрын

Glad it was helpful!

The quality of the content you provide is top-notch and is incomparable with some of the paid courses I've taken before. Thanks a lot for the effort you put into making these amazing videos. :)

@statquest

3 жыл бұрын

Much appreciated!

This video is really awesome !!! I wish I have a teacher like you ! Yours explanations are so clear ! Thank you very much for sharing your knowledge !!!

just amazing, how you clearly put across so complex concepts with ease..!

@statquest

4 жыл бұрын

Thanks! :)

I've always been a bit scared of statistics, but you always make these concepts much easier to understand. Thanks!

@statquest

11 ай бұрын

Happy to help!

Explained it so much better than a 2 hour lecture. The concept makes more sense now.

@statquest

11 ай бұрын

Thanks!

Just incredible!! The best explanation of PCA ever!!! And I'm not even a native speaker of English while I can still understand you better than all the other PCA material written in my mother tongue!

@statquest

4 жыл бұрын

Awesome!!! Thank you very much. :)

I love the exitement of the "BAM!" moments, thanks really helpful

I'm probably going to have to watch this video a couple more times to fully understand everything but this is really cool and very digestible. I was still able to understand the main concept from the first watch. Thank you Josh!

@statquest

3 жыл бұрын

Awesome, thank you!

This is the best explanation out there. Now I understand how the images/audio are compressed with very little loss. Thank you

@statquest

Жыл бұрын

BAM! :)

Man, this work of yours is pure gold. can't thank you enough

@statquest

5 жыл бұрын

Thanks! :)

Tear really flows from my eyes after I watched this video. I learned several hours from my professor but almost did not understand. But you could teach me in under 30 minutes.

@statquest

4 жыл бұрын

Wow! Thank you! :)

The ability to just explain complexe idea is just amazing! Thank you !

@statquest

3 жыл бұрын

Thanks!

It is very hard for me to understand the content in classes, but it is much easier to understand what you taught. You are a lifesaver!

@statquest

2 жыл бұрын

bam!

Clearest explanation of PCA to date! better than all university lecturers who usually just regurgitate jargons that students dont appreciate!

@statquest

3 жыл бұрын

Thanks! :)

one of best interpretation of PCA. It is like turning several math techniques into one idea.

@statquest

4 жыл бұрын

Thank you! :)

The way you explain these supposed complex topics with ease and in a very interactive way is just out of the world! Best content out there ❤❤

@statquest

4 ай бұрын

Thank you very much!

My jaw dropped after watching such a fantastic explanation. Truly amazing!! The intuitive build-up and pacing are so good.

@statquest

9 ай бұрын

Wow, thank you!

You have no idea how grateful I am!!! Your channel is the best!

@statquest

6 жыл бұрын

Thank you so much!!! I'm glad you like the channel. :)

The best PCA explanation ever! Plus a linear algebra bonus!!! Thanks a lot!

@statquest

5 жыл бұрын

Thanks! :)

Simply this is the best explanation of PCA on entire KZread platform, After this you don't need to get anywhere

@statquest

3 жыл бұрын

Bam!

i genuinely love this and you- im a researcher at hopkins in epidemiology- trying to reconcile my biotech + epi trainings using data science and this makes me so HAPPY!!!!!

@statquest

Жыл бұрын

Hooray! :)

i have for the first time in my life visualized eigenvalues and eigenvectors. THANKS MY SAVIOR

@statquest

4 жыл бұрын

Hooray!

I have a cognitive science exam in one hour, your explanation saved me lots of time, so I wanted to thank you so very much before going.

@statquest

3 жыл бұрын

Thanks and good luck with your exam! Let me know how it goes. :)

Not one resource explained the perpendicular line. Thank you so much!! This is so great. I haven't even finished the video yet and know more now about PCA than a year in school!

@statquest

2 жыл бұрын

Glad it was helpful!

Dude, you did not just explain eigenvectors! I have spent years bumping into this term, googling it, not really getting it, and moving on. That is the first explanation that ever made sense, and its not even what I came here for. Thank you so much.

@statquest

Жыл бұрын

bam! :)

Thank you sooooo much. Before I was struck with the amount of math involved and I couldn't understand why we were doing certain things. But this video cleared all my conceptual doubts. TYSM....

@statquest

5 жыл бұрын

Hooray!!! I'm glad to hear the video is helpful.

genius you are in the presentation of your knowledge...

@statquest

5 жыл бұрын

Thank you!

Thanks! I am a German high school student and we don't learn this in school. I stumbled across this video and have to say, I was able to follow everything and learned a lot. Keep up the great work!

@statquest

Жыл бұрын

Wow!!! Thank you so much for supporting StatQuest! BAM! :)

Learning PCA was so daunting before watching this video. I have come back re-watching couple of times when every time I got myself stuck in reading other text books. The really amazing thing for me from your work is the development of intuitions on rather abstract concepts. I don't know about others but my limited brain just can not cope with things that are too abstract and lack of visual cues.... Thank you Mr. Starmer.

@statquest

2 жыл бұрын

I'm glad the video is helpful!

@minweideng4595

2 жыл бұрын

@@statquest I'm also following your recommendation, reading An Introduction to Statistical Learning, which as you said is a great book but not the easiest to read. Your videos help in great deal. Appreciate your knowledge sharing.

@statquest

2 жыл бұрын

@@minweideng4595 BAM!!!

All i can say after watching the video is "Double BAM"😂

Have a test on my Data Science course tomorrow, I was so lost with how to visualise what we are doing in every step and this video helped me so much!

@statquest

3 жыл бұрын

Best of luck on the exam. Let me know how it goes. :)

I think that this is by far the best explanation of a sophisticated statistical concept that I have every seen, bravo!

@statquest

2 жыл бұрын

Thank you!

Hi Josh, You are the gold standard that all teachers from all spheres should aspire to be.

@statquest

2 жыл бұрын

bam! :)

Excellent video! Seriously, you are the compass of biologists trying to navigate through the sea of biostatistics!

@statquest

5 жыл бұрын

Thank you! :)

Wow, we should all just start studying solely with yt. There simply is no better way to understand complex stuff than with a nice video like thise one. Thx

@statquest

2 жыл бұрын

Thank you! :)

Josh, I always feel that you are the creator of all these ML methodologies because you have very good understanding and making them even more simpler while explaining..... Cheers,

@statquest

2 жыл бұрын

Thank you!

Explained very well, Starmer! I'm about to attend a Data Science Interview on 14th of this month. Your videos have been extremely useful for me to revise all concepts. I went through many comments under this video and it is amazing to see that you are taking time to reply to each and every comment! Even as a small youtuber, I'm not able to do that! Hats off man! I see that a few people are doing super thanks. 30% of that will go to KZread. I will definitely come back and support statquest when I get a job soon. But not in the form of Super Thanks!

@statquest

6 ай бұрын

Thank you and good luck with your interview! BAM! :)

One of the best explanations I’ve seen hands down. Thank you

@statquest

4 жыл бұрын

Thanks! :)

Glad I found your channel this month! Looking forward in inhaling all of your content :D It's a very great addition to the very emotionless lectures of my prof..

@statquest

3 жыл бұрын

Awesome, thank you! And thank you for your support!!!

I am a cloud architect taking an MIT ADSP course. If I see things are overwhelming there, I come and watch Statquest. What I like the best about Statquest is how you explain terminology alerts in simple layman's terms. Your channel is something I am more than happy to support. Indeed this is a great service.

@statquest

4 ай бұрын

Thank you very much!

Man believe me you are the best. You make it easier. It is totally different in your teaching philosophy. I really recommend it for guys who don't know anything about PCA. Your attitude in teaching is fantastic.👌👌

@statquest

3 жыл бұрын

Wow, thanks!

This video can not be recommended enough! BAM!! the best PCA explanation in youtube BAM!!

@statquest

4 жыл бұрын

Thank you! :)

Thanks for this video and I finally passed my third actuarial exam (exam srm) on the last Sep. Your work is much appreciated and please keep it going!

@statquest

3 жыл бұрын

Congratulations!!! That's awesome. I'm glad my video was helpful. :)

I like your simple slide with big text and visualisation. Added with your clear, slowly, emphasis the term voice. Perfect!

@statquest

2 жыл бұрын

Awesome, thank you!