Neural Networks Pt. 2: Backpropagation Main Ideas
Backpropagation is the method we use to optimize parameters in a Neural Network. The ideas behind backpropagation are quite simple, but there are tons of details. This StatQuest focuses on explaining the main ideas in a way that is easy to understand.
NOTE: This StatQuest assumes that you already know the main ideas behind...
Neural Networks: • The Essential Main Ide...
The Chain Rule: • The Chain Rule
Gradient Descent: • Gradient Descent, Step...
LAST NOTE: When I was researching this 'Quest, I found this page by Sebastian Raschka to be helpful: sebastianraschka.com/faq/docs...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying my book, The StatQuest Illustrated Guide to Machine Learning:
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZread Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
3:55 Fitting the Neural Network to the data
6:04 The Sum of the Squared Residuals
7:23 Testing different values for a parameter
8:38 Using the Chain Rule to calculate a derivative
13:28 Using Gradient Descent
16:05 Summary
#StatQuest #NeuralNetworks #Backpropagation
Пікірлер: 519
The full Neural Networks playlist, from the basics to deep learning, is here: kzread.info/dash/bejne/daWDyMttYa_MdNo.html Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@motherisape
2 жыл бұрын
Bamm
@gbchrs
2 жыл бұрын
@seanleith5312
11 ай бұрын
Quit the singing, please
@statquest
11 ай бұрын
@@seanleith5312 Noted
Man, your promotions are not shameless! Actually, what you do is a gift for us, for the price that you charge and for the level of the content, we are being gifted and not a buying something. You are far better than a lot of paid (and expensive) courses. Just check out your video comments to see how people few happy when they discover your videos!! Great work as always. Thank you so much!!!👏🏻👏🏻👏🏻👏🏻
@statquest
3 жыл бұрын
Thank you very much! :)
@Luxcium
Жыл бұрын
He is using the concept of reverse psychology by presenting great stuff at a good price and as you mentioned theses promotions are not shameless… They are shameful, as you hinted he should indeed be ashamed of giving us such a good and advantageous offer… 😅😅😅😅
You released this video just in time for my AI exam! Thank you. Sometimes I think professors use really complex notation just to feel smarter than students, it doesn't help learning. I love your content.
@statquest
3 жыл бұрын
Thank you very much!
@sarazahoor9133
2 жыл бұрын
I want to copy-paste this comment! :D
@puppergump4117
2 жыл бұрын
Ain't that right. They must be mad that they don't understand the actually smart people so they don't want to be understood either.
@zhongtianjackwang5346
Жыл бұрын
lol, that is exactly what I want to say
I have been a student my entire life and have taught college level courses myself, and I must say you are one of the finest lecturers I have ever seen. This statquest is a gem. Your work is so succinct and clear its as much art as it is instruction. Thank you for this incredible resource!
@statquest
2 жыл бұрын
Thank you very much! :)
Finally. The wait is overBAM!!!
@statquest
3 жыл бұрын
TRIPEL BAM!!!
@mariolira9279
2 жыл бұрын
F I F T H B A M!
@syco_Rax
2 жыл бұрын
SUPER BAM!!!
I just want you to know your channel has been instrumental in helping me towards my Data Science degree, I'm currently in my last semester. I'll be forever grateful for your channel and the time you take to make these videos. Thank you so much.
@statquest
2 жыл бұрын
Thank you and good luck with your final semester! BAM! :)
omg yay! I just discovered that you've made a million videos on ML. I'm going to go binge all of them now :D
@statquest
3 жыл бұрын
Hope you enjoy!
@motherisape
2 жыл бұрын
Bamm
I am studying for a Master's degree in bioinformatics now, and as someone who knows little about statistics, I really can't thank you enough for your videos and the effort that you have put into them.
@statquest
Жыл бұрын
Thank you!
Love this! You've explained it far better than anywhere else I've seen, and you made it entertaining at the same time! Thank you so much for making this.
@statquest
3 жыл бұрын
Awesome, thank you!
You have no idea how much I appreciate your work. Your explanations are so fun and simple, I'm just so grateful!
@statquest
3 жыл бұрын
Thank you very much! :)
I finished business school 25 years ago where I studied statistics and math. So happy to see that neural networks are fundamentally just a (much) more advanced regression analysis.
@statquest
7 ай бұрын
BAM!!! Thank you for supporting StatQuest! Yes, neural networks are a lot like regression, but now we can fit non-linear shapes to the data, and we don't have to know in advance what that shape should be. Given enough activation functions and hidden layers, the neural network can figure it out on its own.
this video is a gem. I had to watch it a few times and like in gradient descent, I went closer to the target level of knowledge with each step :)
@statquest
12 күн бұрын
BAM! :)
Now Iam reading backpropagation, I worried about this vedio didn't came for long time , And finally I got a treasure.
@statquest
3 жыл бұрын
bam! :)
This is excellent stuff! As a visual learner, your channel is a BLESSING. Thank you so much for your fantastic work on breaking down concepts into small, bite-sized pieces. It's much less intimidating, and you deserve so much more appreciation . You also gained my subscription to your channel! Keep doing a great job, and thank you SO MUCH for having my back!
@statquest
Жыл бұрын
Thank you very much!!! :)
Finally a proper, detailed, step by step explanation. This guy is absolutely AMAZING ! Thank you so much for all the hard work in putting these videos together for us.
@statquest
Жыл бұрын
Thank you very much! :)
Amazing explanation! I've spent years trying to learn this and it always went too quickly into the gory mathematical details. Aha moment for me was when green squiggle equal blue plus orange squiggles lol Thank you for this Josh!!!
@statquest
3 жыл бұрын
Glad it was helpful!
@amandak1396
3 жыл бұрын
Kind of like how Feyman reduced gory math in physics to actual squiggle, double bam!
Josh, this is amazing. You really make things so easy to visualise which is crazy considering the hidden networks are meant to be so hard that they are referred to as black box! Thanks for all your videos. I have used heaps over the last twelve months. Thank you again.
@statquest
3 жыл бұрын
Hooray!!! I'm so glad that you like my videos. :)
JUST WOW! Thank you so much, Josh! I cannot express the feeling I had when EVERYTHING made sense!!! TRIPLE BAM! Never thought I would be extremely excited to pause the video and try to solve everything by hand before I look at the next steps
@statquest
3 жыл бұрын
BAM! :)
@NadaaTaiyab
2 жыл бұрын
oh that's a good idea!
You are the best. I wish every ML learner find you first. I am going to do my part and tweet about you. Thanks for making these videos! Wish you more success.
@statquest
2 жыл бұрын
Wow! Thank you very much! I really appreciate the support. BAM! :)
My Machine Learning exam is tomorrow and re-watching your videos to review concepts is helping me so much! Thank you!!!
@statquest
Жыл бұрын
Good luck! BAM! :)
I just iterated on a gradient descent and found that this is the best possible way to teach this topic and no other lecture in the entire existence is better than this one
@statquest
4 ай бұрын
bam!
May I say .... You are such a good teacher that it is most enjoyable to watch your videos. I am proficient in statistics (via university econometrics 101) ... and I did not realise all those fancy terms in machine learning are actually concepts that are common items in the stats that I learned in the 1970s, e.g., biases and weights, label, activation functions etc. Anyway, I can see that a lot of viewers appreciate your work and teaching. I have also 'updated' myself. Thank you.
@statquest
2 жыл бұрын
Thank you very much!
Thank you so much for this series! I haven’t been able to find any other videos that really break down the mechanics of neural networks like this.
@statquest
Жыл бұрын
Thanks!
Teaching such complicated topics in a simple, Easily Understandable way.👏👏👏.Thank you, Professor
@statquest
Жыл бұрын
Thanks!
Omg, protect this man at all costs, this was pure gold!!! Also, thank you, sir, for talking so slowly because if my brain squiggles need to work faster they will burn up x)
@statquest
3 жыл бұрын
Glad you enjoyed it!
Wow! This is an incredible video. Thank you SO MUCH for making this for us. This is one of the best videos I've seen to explain this concept. The hard work you have put into this is something that I am incredibly appreciative of. Thanks, man.
@statquest
Жыл бұрын
Wow, thank you!
Its the teacher who makes the Subject easy or difficult and the way you explained Neural Network, I am speechless
@statquest
Жыл бұрын
Thanks!
How amazing is the way you convey complex concepts.
@statquest
2 жыл бұрын
Thank you!
Incredible. Sometimes I need a refresher on these topics. There’s much to remember as a data scientist. I’m so glad I found your channel!
@statquest
3 жыл бұрын
Bam!
Wow Josh way to go!!!! You have the concepts so clear in your own head that it seems a piece of cake for us 🍰♥️ Love from India! 🇮🇳
@statquest
2 жыл бұрын
Thanks so much!!
I am really glad that people like you exist!! Thank you so much for those incredible lessons
@statquest
10 ай бұрын
Glad you like them!
Best explanation that I've seen so far on backpropagation!
@statquest
3 жыл бұрын
Thank you! :)
Excellent job Josh!!! Just loved it!!! Thanks a ton for your fun-filled tutorials :)
@statquest
3 жыл бұрын
Glad you like them!
Thank you for this video. I have been waiting for this all the time. Finally, it appeared just 1 day before my exam. You are a life saver!!
@statquest
3 жыл бұрын
Good luck with your exam! :)
This is really awesome. I thank you for your effort in developing this highly enriched content. BAM !!!
@statquest
3 жыл бұрын
Thank you!
I searched neural net and again your video popped that too just 4 month old, love to get your helpful videos right before my semester
@statquest
3 жыл бұрын
:)
For the first time ever in history, I have understood the concept behind Neural Networks! BAM!!!! :D Thanks Josh, so grateful :)
@statquest
2 жыл бұрын
BAM! :)
I never understood backpropagation. I knew some things from other tutorials, but as for beigginer, it was very hard to understand. This video (and probably series) is the best i could find. Thank you.
@statquest
2 жыл бұрын
Glad it was helpful!
I . JUST . LOVE . YOUR . CHANNEL !! you literly explain things very clearly and step by step! I just cannot thank you enough really
@statquest
Жыл бұрын
Wow, thank you!
I didn't even get through the jingle before I gave a thumbs up. Thanks for the chuckle, can't wait to watch the rest of this!
@statquest
Жыл бұрын
BAM! :)
I found your explanation is far more easier to understand than the edx online course I am taking, BAM !!!
@statquest
2 жыл бұрын
bam!
Honestly you do a much better job at teaching using a pre recorded video than my instructors using both written and live materials that I'm paying for.
@statquest
3 ай бұрын
I'm glad my videos are helpful! :)
The time couln't be more perfect ... 3 hours before my machine learning exam !! Thank you!!!!
@statquest
3 жыл бұрын
Good luck with your exam! I hope it goes well.
The type of in depth video I’ve always wanted!
@statquest
2 жыл бұрын
Thank you!
I've been waiting for this all the time checking the notification haha
@statquest
3 жыл бұрын
Hooray! The wait is over.
Best Neural Networks Lectures! Just ordered the book from Amazon to support!
@statquest
Жыл бұрын
Wow! Than you very much! :)
Lecture was awesome,more affective and easy to understand Thanks
@statquest
Жыл бұрын
Thank you! :)
This video made my day, thank you so much, Josh!!
@statquest
2 жыл бұрын
Thanks!
Hello sir, Thanks for such an simple explanation, never understood back propagation in such a depth at this ease. 🎉
@statquest
10 ай бұрын
Thank you!
The level of detailing in this video is just 🤯 I hope i can try to make my Quantum Computing videos this clear!
@statquest
4 ай бұрын
Good luck!
Who else is using these videos to put together a semester project? So far, I've put Regression Trees, K-fold CV, complexity pruning, and now Neural networks as my final model construction. Josh is worth a double bam every time.
@statquest
3 жыл бұрын
BAM! Good luck with your project.
Your getting better and better. Thank you
@alexissanchezbro
3 жыл бұрын
BAAAAAAAMMM
@statquest
3 жыл бұрын
:)
Thank you. Now everything is more clear.
@statquest
3 жыл бұрын
BAM! :)
Excellent content, excellent delivery - just bought your book!
@statquest
5 ай бұрын
Thank you so much for supporting StatQuest! BAM! :)
Thank you for the amazing effort you put into this video and BAM!!! as always!
@statquest
Жыл бұрын
Thanks!
You rescued me from the unknown!! Much Love from Ethiopia
@statquest
Жыл бұрын
Bam! :)
9:00 at this moment I realised I'm watching the best math content on earth, because you never see simple stuff like this being given attention to. Luckily I already know how summation symbol works, but I didn't know it in the past, and nobody cared to explain. But it's just not about the summation symbol, imagine the other 1000 small things somebody might not understand, and doesn't realise they don't understand, because it's been skimmed over
@statquest
Жыл бұрын
Thank you so much! I really appreciate it! :)
BAM! Thanks Josh! You’re the best! Got myself a pink T-shirt 😍😍😍
@statquest
3 жыл бұрын
Hooray! And thank you for supporting StatQuest!!!
You have cleared my doubt on back propagation, gradient descent and chain rule. Triple Bam !
@statquest
7 ай бұрын
:)
Another great one. Thanks!
@statquest
8 күн бұрын
Thanks again!
perfect you explain complicated things very underatandable it's amazing
@statquest
2 жыл бұрын
Thank you very much! :)
insane teaching quality, thanks !
@statquest
Жыл бұрын
Glad you think so!
You did a great job!. Thank you so much
@statquest
2 жыл бұрын
Thanks!
Thanks Josh! you simply the best
@statquest
3 жыл бұрын
Thank you very much. I can't wait to get the other videos out soon.
Thanks a lot Sir, was waiting for this
@statquest
3 жыл бұрын
Bam! :)
That Josh guy is just awsome !! Definitely will support !!
@statquest
Жыл бұрын
Thank you!
Thank you very much for your video~ Your videos make me feel that studying English make so much sense, otherwise I can't enjoy such beautiful thing~ 👍👍👍❤❤❤
@statquest
2 жыл бұрын
WOW! Thank you very much!!! And thank you for your support!!! :)
so good. can't wait for the next one!
@statquest
3 жыл бұрын
Bam! It should be out soon.
I was reading an article based on Backpropagation and I did not understand a single word. I had to watch all your videos starting from Chain Rule, Gradient Descent, NNs...I re-read the article and understood everything!!! But now I can't get the beep--boop and small/double/triple/ bam out of my head lol.
@statquest
Жыл бұрын
BAM! I'm glad my videos were helpful! :)
This is finally all making sense to me thank you
@statquest
10 ай бұрын
Thanks!
Perfect explanation!
@statquest
2 жыл бұрын
Thank you!
give respect to my math teacher, but this is so much easier to understand.
@statquest
2 ай бұрын
bam! :)
I learn more from four of your videos than 200 pages of textbook gibberish
@statquest
2 жыл бұрын
Thanks!
This is amazing thank you very much!
@statquest
2 жыл бұрын
Glad you like it!
You got the best way to teach thank you❤
@statquest
Жыл бұрын
Thanks!
Awesome lecture
Wow 😮 I didn't knew I had to watch the *Gradient Descent Step-by-Step!!!* before I can watch the video related to *Neural Networks part 2* that I must watch before I can watch the *The StatQuest Introduction To PyTorch...* before I can watch the *Introduction to coding neural networks with PyTorch and Lightning* 🌩️ (it’s something related to the cloud I understand) I am genuinely so happy to learn about that stuff with you Josh❤ I will go watch the other videos first and then I will back propagate to this video...
@statquest
Жыл бұрын
Getting warmer...
great explanation sir. Tripple BAM........kudos to your presentation.
@statquest
2 жыл бұрын
Thank you! :)
Great video !
@statquest
2 жыл бұрын
Thank you!
Unbelievable! Thank you!!
@statquest
3 ай бұрын
Thanks!
Thanks a lot, Josh!
@statquest
Жыл бұрын
My pleasure!
I wish our lecturers watched these videos before trying to make their own teaching slides! (With acknowledgement of course!)
@statquest
3 ай бұрын
bam!
You always are the best.
@statquest
Жыл бұрын
Thanks!
u r amazing! ur effort r so impressive!
@statquest
2 жыл бұрын
Thank you so much 😀
Thanks man❤️
@statquest
3 жыл бұрын
:)
I'm buying the shirt! Kudos to you sir.
@statquest
3 жыл бұрын
Awesome! Thank you!
your channel is amazing...
@statquest
2 жыл бұрын
Thanks!
very informative ty
@statquest
Жыл бұрын
:)
amazing content
@statquest
Жыл бұрын
Thank you!
Saludos desde San Cristóbal, Venezuela!!!
@statquest
Ай бұрын
:)
Thank you so much
@statquest
9 ай бұрын
No problem!
The understated BAM at 4:40 cracked me up 😂
@statquest
3 жыл бұрын
SPLICY in the house!!! BAM! :)
I really like your lessons guys, way2go
@statquest
3 жыл бұрын
Thanks!
Thanks for the great video!! My notes: 7:23 8:11 8:48 10:00 10:22❗,11:13 - 11:48, 11:56 12:08 13:30❗,
@statquest
10 ай бұрын
BAM! :)
Thank you from France
@statquest
3 жыл бұрын
Merci! :)
Where were you 5 years ago???!?!?! :D Awesome work man! Keep it up :)
@statquest
3 жыл бұрын
Thanks! I have 4 more neural network videos coming out in the next month.
@richarda1630
3 жыл бұрын
@@statquest awesome! can't wait :D
Great explanatioh! Do you think you'd make videos about more advanced neural networks in the future?
@statquest
3 жыл бұрын
That's the plan.
Much waiting.... Finally came..
@statquest
3 жыл бұрын
Bam! :)
@SPLICY
3 жыл бұрын
This is what she said
Back propagation (aka finding w's and b's) start with b_final=0. you'll notice that error = (actual - predicted)^2 is really high. so you find the gradient descent of squared error wrt b_final and find out the value of b_final for which the squared error is minimum. that is your optimal b_final. gradient descent: derivative of sum of squared errors wrt b_final = derivative of sum of squared errors wrt predicted value y * derivative of y wrt b_final. d(y observed - y predicted)^2/d(y predicted) = -2*(y observed - y predicted) d(y predicted)/d(b_final) = d(sum of all those previous curves obtained through each node of the layer + b_final)/d(b_final) = 0+0+0....+0+1=1 take the predicted curve ke x values and find the derivative/slope. step size = slope*learning rate. new b_final = old b_final - step size. keep repeating until slope touches 0. this is how gradient descent works and you've found your optimal b_final.
@statquest
7 ай бұрын
double bam
Thank you king
@statquest
3 жыл бұрын
:)