Essential Matrix Algebra for Neural Networks, Clearly Explained!!!

Although you don't need to know matrix algebra to understand the ideas behind neural networks, if you want to code them or read the latest manuscripts about the field, then you'll need to understand matrix algebra. This video teaches the essential topics in matrix algebra and shows how a neural network can be written as a matrix equation, and then shows how understand PyTorch documentation, error messages and the equations for Attention, which is the fundamental concept behind ChatGPT.
Note: If you want to learn more about neural networks...
• The Essential Main Ide...
...backpropagation...
• Neural Networks Pt. 2:...
...the ReLU activation function...
• Neural Networks Pt. 3:...
...tensors...
• Tensors for Neural Net...
...SoftMax...
• Neural Networks Part 5...
...Transformers and Attention...
• Transformer Neural Net...
If you'd like to support StatQuest, please consider...
Patreon: / statquest
...or...
KZread Membership: / @statquest
...buying my book, a study guide, a t-shirt or hoodie, or a song from the StatQuest store...
statquest.org/statquest-store/
...or just donating to StatQuest!
paypal: www.paypal.me/statquest
venmo: @JoshStarmer
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
2:35 Introduction to linear transformations
5:57 Linear transformations in matrix notation
7:34 Matrix multiplication
11:03 Matrix multiplication consolidates a sequence of linear transformations
13: 46 Order matters for matrix multiplication
15:18 Transposing a matrix
16:37 Matrix notation and equations
18:51 Using matrix equations to describe a neural network
24:26 nn.Linear() documentation explained
26:38 1-D vs 2-D error messages explained
27:17 The matrix equation for Attention explained
#StatQuest #neuralnetworks #matrixalgebra

Пікірлер: 134

  • @statquest
    @statquest6 ай бұрын

    To learn more about Lightning: lightning.ai/ Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

  • @bigbacktor

    @bigbacktor

    5 ай бұрын

    Hi, i would like to buy the book in color. Does someone know if that is posible ? It seems to me that in amazon is in black and white and on Lulu is on color. Is that right ? I am from spain.

  • @statquest

    @statquest

    5 ай бұрын

    @@bigbacktor They are all in color. And there is a version that is translated into spanish if you are interested.

  • @bigbacktor

    @bigbacktor

    5 ай бұрын

    @@statquest Thanks a lot!! BAM you just sold another book :)

  • @statquest

    @statquest

    5 ай бұрын

    @@bigbacktor Hooray!!! Thank you very much for supporting StatQuest! BAM! :)

  • @user-gx9hk8gt3k
    @user-gx9hk8gt3k6 ай бұрын

    You have been staying the course for a long, long time. It's not so easy ! Keep up the good work!

  • @statquest

    @statquest

    6 ай бұрын

    Thank you!

  • @user-do2pr3ue9x
    @user-do2pr3ue9x6 ай бұрын

    What you do is nothing short of a miracle! Immense gratitude

  • @statquest

    @statquest

    6 ай бұрын

    Thank you!

  • @bin4ry_d3struct0r
    @bin4ry_d3struct0r6 ай бұрын

    All visual learners are blessed by the great Josh Starmer!

  • @statquest

    @statquest

    5 ай бұрын

    Thank you! :)

  • @kitkitmessi
    @kitkitmessi2 ай бұрын

    this is freaking amazing! Would love to see more math lessons like this

  • @statquest

    @statquest

    2 ай бұрын

    Thank you!

  • @Why_I_am_a_theist
    @Why_I_am_a_theist6 ай бұрын

    World is a better place with Josh🎉

  • @statquest

    @statquest

    6 ай бұрын

    Thanks!

  • @michaelzap8528
    @michaelzap852819 сағат бұрын

    Best of the best. I'm really speechless now.

  • @statquest

    @statquest

    15 сағат бұрын

    Thank you!

  • @haitematik5832
    @haitematik58323 ай бұрын

    Man u deserve a thousand times more subscribers

  • @statquest

    @statquest

    3 ай бұрын

    Thank you!

  • @free_thinker4958
    @free_thinker49585 ай бұрын

    Could you please make a video about QLORA? ❤ You're our savior when it comes to understanding complex concepts, thank you man

  • @statquest

    @statquest

    5 ай бұрын

    I'll keep that in mind.

  • @TheTruthOfAI

    @TheTruthOfAI

    4 ай бұрын

    that would be lovely, perhaps LORA itself holds a strong glue potential across neural networks, will be looking forward for such amazing video

  • @varshinibalaji3535
    @varshinibalaji35355 ай бұрын

    Wow, just the perfect video I was looking for! Loved all the Taylor references, and music puns.

  • @statquest

    @statquest

    5 ай бұрын

    Hooray!!! You're the first person to mention the Taylor references in a comment. BAM!!! :)

  • @varshinibalaji3535

    @varshinibalaji3535

    5 ай бұрын

    @@statquest I was looking for it, since you mentioned something with Taylor was coming soon, in one of my Linkedin Posts :) Plus, I've been reading a lot of academic papers lately, So needed a better context on matrix transformations to interpret the math better! So, Double BAM, indeed!

  • @user-cf8rq6wk8w
    @user-cf8rq6wk8w5 ай бұрын

    Hi Josh... would you please make a video and explain the differences between different statistical tests like t, z, chi... I want to know the differences and when to use each.

  • @statquest

    @statquest

    5 ай бұрын

    I'll keep that in mind.

  • @ageofkz
    @ageofkz3 ай бұрын

    Really amazing work! This set of videos (neural network playlist) has really helped me in my uni coursework and project! My groupmates and I are planning to get a statquest triple bam hoodie each haha!

  • @statquest

    @statquest

    3 ай бұрын

    That's awesome!!! TRIPLE BAM! :)

  • @anmolarora2599
    @anmolarora25993 ай бұрын

    Thank you for explaining it so simply even a novice like me can understand it.

  • @statquest

    @statquest

    3 ай бұрын

    Thanks!

  • @TheTruthOfAI
    @TheTruthOfAI4 ай бұрын

    i love your videos, it helped me so much.. learned a lot.. i was able to make UNA thanks to your learnings :)

  • @statquest

    @statquest

    4 ай бұрын

    Triple bam! Congratulations!

  • @ShadArfMohammed
    @ShadArfMohammed6 ай бұрын

    Baaam this is good :D I have been waiting for this, to be honest, I had the feeling that one day you would make such a tutorial. Your content is great.

  • @statquest

    @statquest

    6 ай бұрын

    Thank you very much!

  • @AyumiFortunex
    @AyumiFortunex3 ай бұрын

    Absolutely fantastic explanation again

  • @statquest

    @statquest

    3 ай бұрын

    Thank you!

  • @ritshpatidar
    @ritshpatidar6 ай бұрын

    I was thinking about taking a course to learn matrix algebra yesterday. Thanks for posting this video. It is really helpful and it is like a wish came true.

  • @statquest

    @statquest

    6 ай бұрын

    BAM! :)

  • @arenashawn772
    @arenashawn7725 ай бұрын

    I found this channel when searching for a clear explanation of central limit theorem on Google (after doing some simulation in R using sample size much less than 30 and being intrigued by the results I got) and I just want to say I love the content so much! (And the ukulele episode ❤) I’ve recently started some machine learning classes on coursera and EdX, and I must say the explanation you have here in these episodes are SO MUCH BETTER AND MORE TO THE POINT/BETTER DEFINED than the multi thousand dollar classes (I’m surely glad I chose to audit them first!) taught by professors from Harvard or Engineers working for Google/IBM. So much better!… ❤❤❤ Just want to say thank you and Merry Christmas! I know I will be going through these videos one by one in the coming months…

  • @statquest

    @statquest

    5 ай бұрын

    Thank you very much!!! I'm so happy you enjoy my videos. BAM! :)

  • @arenashawn772

    @arenashawn772

    5 ай бұрын

    ⁠@@statquestI really did and binge watched a bunch… But I must say I now enjoy your songs even more 😂 Just bought all your albums on bandcamp - they are awesome! That going back to Cali song just had me rolling off my chairs at the end of it… I relocated from San Francisco Bay Area to Florida panhandle not long ago so that song really struck a cord with me 😂😂😂

  • @statquest

    @statquest

    5 ай бұрын

    @@arenashawn772 Thank you very much! I'm glad you enjoy the tunes and the videos. I hope the move went well! :)

  • @randr10
    @randr106 ай бұрын

    Thank you for this video. I think I understand what a transformer is now.

  • @statquest

    @statquest

    6 ай бұрын

    Thanks!

  • @ilirhajrullahu4083
    @ilirhajrullahu40836 ай бұрын

    Very nice video! Thank you for uploading such helpful material :). It would be great if you made a video on vector and matrix calculus. These are important topics in NNs too :).

  • @statquest

    @statquest

    6 ай бұрын

    I'll keep that in mind.

  • @gocomputing8529

    @gocomputing8529

    5 ай бұрын

    Thanks for the great video! Also the topic proposed here would also be super interesting, so I hope you could do it someday

  • @kartikchaturvedi7868
    @kartikchaturvedi78685 ай бұрын

    Superrrb Awesome Fantastic video

  • @statquest

    @statquest

    5 ай бұрын

    Thanks 🤗

  • @BrianPondiGeoGeek
    @BrianPondiGeoGeek6 ай бұрын

    Tripple Bam for sure. Amazing explanation.

  • @statquest

    @statquest

    6 ай бұрын

    Thanks!

  • @louisnemzer6801
    @louisnemzer68016 ай бұрын

    Squatch: So it's all just matrix multiplication? Josh: Always has been

  • @statquest

    @statquest

    6 ай бұрын

    bam! :)

  • @techproductowner
    @techproductowner3 ай бұрын

    You will be known and remembered for the next 1000 years ..

  • @statquest

    @statquest

    3 ай бұрын

    bam!

  • @harryliu1005

    @harryliu1005

    2 ай бұрын

    10000000 years

  • @siddhanthbhattacharyya4206
    @siddhanthbhattacharyya42062 ай бұрын

    Quadruple bam! (One bam for me finally understanding)

  • @statquest

    @statquest

    2 ай бұрын

    Hooray! :)

  • @kujohjotaro3017
    @kujohjotaro30175 ай бұрын

    Your video is just a lifesaver to me and my essay! Could you make a video on the Glove model in NLP?

  • @statquest

    @statquest

    5 ай бұрын

    I'll keep that in mind.

  • @NJCLM
    @NJCLM6 ай бұрын

    Very good video ! You should remake one of the transformer videos with the matrix notation as you done at the end of this vide.

  • @statquest

    @statquest

    6 ай бұрын

    I'm working on it right now. Hopefully it will be ready soon.

  • @NJCLM

    @NJCLM

    6 ай бұрын

    @@statquest take your time and thanks you very much, your content is so much valuable !

  • @slash_29
    @slash_296 ай бұрын

    Please part 2 with more details, and new terms

  • @statquest

    @statquest

    6 ай бұрын

    I'll keep that in mind.

  • @amjadiqbal478
    @amjadiqbal4782 ай бұрын

    Quite good. ❤

  • @statquest

    @statquest

    2 ай бұрын

    Thanks!

  • @anonymousgirl1463
    @anonymousgirl14635 ай бұрын

    Hey Josh! I love your channel and I was thinking about buying a study guide. What is the difference between watching one of your playlists and buying a study guide? Do you cover exactly the same in both and buying the study guide is for support/like a donation or is there any difference?

  • @statquest

    @statquest

    5 ай бұрын

    They are the same. The difference is that some people like to have the study guides for offline use or adding their own notes to. In some ways, the study guides are like "cheat sheets" - everything in a video is condensed to about 3 to 5 pages.

  • @PavanKumar-pt2sh
    @PavanKumar-pt2sh6 ай бұрын

    Can you please create a video on multi-modal transformer architecture?

  • @statquest

    @statquest

    6 ай бұрын

    I'll keep that in mind.

  • @free_thinker4958

    @free_thinker4958

    5 ай бұрын

    I hope he does it, he's our savior when it comes to understanding complex concepts

  • @Ahmed_Issaoui
    @Ahmed_Issaoui5 ай бұрын

    hello statquest, what software do you use to create your videos ? (your answer is really useful to me)

  • @statquest

    @statquest

    5 ай бұрын

    I give away all of my secrets in this video: kzread.info/dash/bejne/laaAuqyAXainmM4.html

  • @ivant_true
    @ivant_true6 ай бұрын

    great

  • @statquest

    @statquest

    6 ай бұрын

    Thanks!

  • @pran441
    @pran4415 ай бұрын

    Joshua your teaching was fantastic, but I couldn't quite grasp the concept.

  • @statquest

    @statquest

    5 ай бұрын

    What time point (minutes and seconds) was confusing?

  • @ps3301
    @ps33016 ай бұрын

    Could you explain the math behind a basic liquid neuron and show how it differs from other neuron ?

  • @statquest

    @statquest

    6 ай бұрын

    I'll keep that in mind.

  • @nivcohen961
    @nivcohen9613 ай бұрын

    You are awsome

  • @statquest

    @statquest

    3 ай бұрын

    Thanks!

  • @magtazeum4071
    @magtazeum40714 ай бұрын

    Hi Josh, Could you do video on time series clustering , and time series analysis please?

  • @statquest

    @statquest

    4 ай бұрын

    I'll keep that in mind.

  • @davidmurphy563
    @davidmurphy5636 ай бұрын

    Ok, it always annoyed me that when you're doing matrix vector (col) multiplication they always write the matrix first, then the vector. It never occured to me until you said so just now that the cols and rows aren't valid tensor operations if you write them the other way round... Doh! It doesn't look nice though. Btw, why did you use a row vector and a transverse matrix? I would always use a col vector. Col space transforms are the default for me and you can picture the latent space. The only times I'd use rows is if I have a system of linear equations.

  • @statquest

    @statquest

    6 ай бұрын

    I agree that the matrix * column looks bad. And I chose to do row * matrix because that is what they used in the PyTorch documentation.

  • @davidmurphy563

    @davidmurphy563

    6 ай бұрын

    @@statquestGlad it's not just me that thinks it looks backwards! :)) But you're of course right; 2x2 * 2x1 is a valid operation whereas 2x1 * 2x2 is, strictly speaking, undefined. Oh, a tip you may (or may not!) find a useful teaching tool: I always look at matrix multiplication in terms of a series of dot product operations. Once the student understands that the dot product outputs a scalar expressing the likeness of two vectors (eg whether two normalised vectors pointing the same way) then rather than just mechanically running an algorithm - the student can see that it's plotting the vector in the new space by comparing its likeness to the space's basis vectors one axis at a time. That's why I think it's always handy to see a square matrix as a series of basis vectors. So, if you're going from an orthonormal basis to one where, say, y is mirrored - {{1, 0}, {0, -1}} - then it's quite apparent why taking the dot product for each spatial dimension will plot the vector upside-down. You could show an image flipping to drive the point home. I just think that's intuitive and why we're multiplying and adding across columns and rows. At least that's how I like to see it.

  • @user-ut3sy6hy4p
    @user-ut3sy6hy4p2 ай бұрын

    thanks for ur effort, ur videos helped me so much, but could u plz tell us how lghm works

  • @statquest

    @statquest

    2 ай бұрын

    Do you mean Light Gradient Boost? LightGBM?

  • @user-ut3sy6hy4p

    @user-ut3sy6hy4p

    2 ай бұрын

    I mean LightGBM @@statquest

  • @deveshbhatt4063
    @deveshbhatt40636 ай бұрын

    Triple Bam🎉❤

  • @statquest

    @statquest

    6 ай бұрын

    YES! :)

  • @user-ob4eh9li9e
    @user-ob4eh9li9e4 ай бұрын

    Can u please discuss about stochastic gradient boosting for classification?. I'm having trouble understanding that 😢

  • @statquest

    @statquest

    4 ай бұрын

    I have a whole series of videos on Gradient Boosting. You can find them here: statquest.org/video-index/

  • @shouvikdey7078
    @shouvikdey70785 ай бұрын

    Do more videos related to GAN etc.

  • @statquest

    @statquest

    5 ай бұрын

    I'll keep that in mind.

  • @jarsal_firahel
    @jarsal_firahel2 ай бұрын

    What about a video on MAMBA architecture ? That would be really BAAAM

  • @statquest

    @statquest

    2 ай бұрын

    I'll keep that in mind.

  • @artofwrick
    @artofwrick2 күн бұрын

    Should we move to kaggle as we learn from these videos? Like solving their exercises to get a feel of the real world machine learning in python to the competitions they host? Or is there anything in between?

  • @statquest

    @statquest

    Күн бұрын

    You can definitely take a stab at a Kaggle dataset see how things go. However, I also have some videos that walk you through how to do real-world data analyses here: kzread.info/dash/bejne/eaZ-spt_fLmZdqg.html and here kzread.info/dash/bejne/anVrrpKAo6XPfLQ.html

  • @vigneshvicky6720
    @vigneshvicky67206 ай бұрын

    We want yolo series mainly yolov8 from scratch

  • @statquest

    @statquest

    6 ай бұрын

    I'll keep that in mind.

  • @anickkhan

    @anickkhan

    5 ай бұрын

    Please Professor, it’s an earnest request. Lots of Love from Bangladesh ❤❤

  • @bossgd100
    @bossgd1005 ай бұрын

    Woaw

  • @statquest

    @statquest

    5 ай бұрын

    :)

  • @aga5979
    @aga597922 күн бұрын

    Could you do a series on "attention is all you need " paper ? Thank you Sir.

  • @statquest

    @statquest

    21 күн бұрын

    This video walks you through the concepts in that paper: kzread.info/dash/bejne/rKyF27aEaNTbqbw.html And this video goes through the math: kzread.info/dash/bejne/faScz7GDnq-nlKw.html

  • @aga5979

    @aga5979

    21 күн бұрын

    @@statquest thank you so much!!

  • @midhileshmomidi3120
    @midhileshmomidi31205 ай бұрын

    Can we book on these concepts as well

  • @statquest

    @statquest

    5 ай бұрын

    I'm writing it right now.

  • @lifeisbeautifu1
    @lifeisbeautifu13 ай бұрын

    BAM!

  • @statquest

    @statquest

    3 ай бұрын

    :)

  • @Nono-de3zi
    @Nono-de3zi6 ай бұрын

    Thanks Josh. But naughty, naughty, the stage is not just rotating, it is flipping. Which you can also encode in matrices of course ;-)

  • @statquest

    @statquest

    6 ай бұрын

    I'm not sure I understand what you mean by flipping in addition to rotating as stage left and stage right are maintained through out each change.

  • @Nono-de3zi

    @Nono-de3zi

    6 ай бұрын

    @@statquest The drawing of the stage is asymmetrical (one edge is slightly erased). When you did the slides you flipped it instead of rotating it. As a result, Statsquatch is sometimes on one side, sometimes on the other. I know it was not on purpose 🙂 Thanks for the excellent vid as usual.

  • @statquest

    @statquest

    6 ай бұрын

    @@Nono-de3zi I'm still confused because statsquach is always on stage left.

  • @Nono-de3zi

    @Nono-de3zi

    6 ай бұрын

    But the *stage* is flipped :-)

  • @statquest

    @statquest

    6 ай бұрын

    @@Nono-de3zi If it was flipped, then wouldn't stage left stay on top and stage right stay on the bottom?

  • @harryliu1005
    @harryliu10052 ай бұрын

    after 10000000 years, scientists found fossil record of statquest, then he said " BAM!"

  • @statquest

    @statquest

    2 ай бұрын

    Ha! You made me laugh.

  • @nutzeeer
    @nutzeeer6 ай бұрын

    14:05 matrix multiplication cant be rearranged, as matrix multiplication is a sequence of calculations. is this indicated by using X as a multillication symbol and not •? Becaus in school we used • to indicate multiplications.

  • @nutzeeer

    @nutzeeer

    6 ай бұрын

    ah no the x is not signifying order. but I would like that to be visible from writing alone, without the helpful explanation.

  • @nutzeeer

    @nutzeeer

    6 ай бұрын

    i wonder why matrices are turned sideways like that. it would feel easier for me to multiply rows with rows.

  • @statquest

    @statquest

    6 ай бұрын

    This is explained, although I'm guessing not to your satisfaction, at 10:58. It has to do with the ability to combine transformations. For more details, see: math.stackexchange.com/questions/271927/why-historically-do-we-multiply-matrices-as-we-do

  • @liuze9280
    @liuze92805 ай бұрын

    Great video, but i don't quite understand 25:25...

  • @statquest

    @statquest

    5 ай бұрын

    It just means that PyTorch stores the weights differently than we used in the earlier examples and in order to get the same math, we have to transpose the PyTorch weight matrix.

  • @werewolfprogrammer
    @werewolfprogrammer5 ай бұрын

    Hi, I am trying to start a youtube channel to make tutorial videos about data science related topics. I want to make the videos about things that are less popular but still important, since I found that it can be quite difficult to start off with these things since most information is in difficult to comprehend papers. My starting point will be social network analysis and natural language processing as that is my main interest and expertise. However, I am interested in finding more topics so I am starting by doing research on different channels that make tutorials for data science, AI, machine learning, statistics, natural language processing, graph theory or network analysis. So for anybody in the comments that reads this message, could you help me out by replying with any youtube creators that do something related to these topics, or any other digital platform like Brilliant. If you know a topic that is similar to the ones I mentioned that would also be a great thing to share. Or if you know of better places to share this message. Or any other helpfull tips. Thanks everybody for the help. If this message is regarded as spam also please say so and I will remove it. The topics again: -data science -AI -machine learning -statistics -natural language processing -graph theory -network analysis

  • @tsunningwah3471
    @tsunningwah34715 ай бұрын

    zhina!

  • @statquest

    @statquest

    5 ай бұрын

    ?

  • @samiotmani9092
    @samiotmani90923 ай бұрын

    97 videos finished … small bam 🥲

  • @statquest

    @statquest

    3 ай бұрын

    Wow!