3.2: Linear Regression with Ordinary Least Squares Part 1 - Intelligence and Learning
In this video, part of my series on "Machine Learning", I explain how to perform Linear Regression for a 2D dataset using the Ordinary Least Squares method. In Part 2, I demonstrate how to code the algorithm in JavaScript, using the p5.js library
This video is part of session 3 of my Spring 2017 ITP "Intelligence and Learning" course (github.com/shiffman/NOC-S17-2...)
Link to Part 2: • 3.3: Linear Regression...
Support this channel on Patreon: / codingtrain
To buy Coding Train merchandise: www.designbyhumans.com/shop/c...
Send me your questions and coding challenges!: github.com/CodingTrain/Rainbo...
Contact:
Twitter: / shiffman
The Coding Train website: thecodingtrain.com/
Links discussed in this video:
Session 3 of Intelligence and Learning: github.com/shiffman/NOC-S17-2...
Nature of Code: natureofcode.com/
kwichmann's Linear Regression Diagnostics: kwichmann.github.io/ml_sandbo...
Linear Regression on Wikipedia: en.wikipedia.org/wiki/Linear_...
Anscombe's quartet on Wikipedia: en.wikipedia.org/wiki/Anscomb...
Source Code for the all Video Lessons: github.com/CodingTrain/Rainbo...
p5.js: p5js.org/
Processing: processing.org
For More Coding Challenges: • Coding Challenges
For More Intelligence and Learning: • Intelligence and Learning
Help us caption & translate this video!
amara.org/v/7UbL/
📄 Code of Conduct: github.com/CodingTrain/Code-o...
Пікірлер: 220
I have linear depression
@112BALAGE112
7 жыл бұрын
Happiness goes straight down.
@extradimension7356
7 жыл бұрын
If you don't find help it can be iterative problem
@kayinb90
4 жыл бұрын
😂 me too lmfao
@MrBollec
4 жыл бұрын
I hope that slope was negative ;)
Subbed for multiple color markers.
@TheCodingTrain
7 жыл бұрын
Mission accomplished!
@csanadtemesvari9251
5 жыл бұрын
Check out blackpenredpen then!
@sii479
4 жыл бұрын
Same
"I'm really sorry, I'm going to do this in Fahrenheit, I have to apologize for that. I'll happily say "maths" though." You crack me up Dan
Great explanation. You're the only channel that has such a nice balance between explaining academic content and programming it.
I am smiling now...I was very confused till i watched your explanation. Thank you Sir!
ordinary least Square .. Very well explained Sir :-) ... i am preparing for data science as my carrier ...
Nice video! The numerator of the expression for the slope is actually the sample covariance of X and Y. Sample covariance is a measure of linear dependence. The denominator is the sample variance of X. So the slope is loosely said the linear dependence of X and Y, normalized by the variability in X. Hope that makes sense!
@TheCodingTrain
7 жыл бұрын
Yes, it does, thank you!!
Nice job re-capping the marker as you said "recap" at 14:22.
@TheCodingTrain
7 жыл бұрын
Hah!
@hamad.learns
5 жыл бұрын
Dude how do you even recognize that !
heehhhe.. bro honestly you have win my day. your sense of humor makes me understand it greatly. congrats again. hehe
Great job! This is the nth video of yours I've watched and you've been able to explain hard concepts like AI algorithms like they were nuts! It seems they're so easy and I don't know if you're too good to explain or I'm so dumb I'm not catching the things right.
Thanks for the video. finally easy explanation of linear regression.
It all makes sense now thank you for saving the day and you make learning fun :)
You’re one of my heroes, man. Seeing you make these coding videos years ago inspired me to learn new languages and technologies. Playing with P5.js is lots of fun.
Dan, you are precisely the one, that our education institutions currently need, simply amazing way of making anyone understand even the most complex stuff
@TheCodingTrain
7 жыл бұрын
Thank you so much!
Great work sir, You have an art of teaching with fun.
one of the best explanation on linear regression! thank you!
I was having trouble understanding this. Thanks to you I get it now!!
Please BIG Please, continue with this series. I've followed along with your genetic algorithms, and now I'm busy with neural networks, so please continue with this. Would really appreciate it.
@TheCodingTrain
7 жыл бұрын
Thanks, I am going to!
You are just awesome. ..so intelligent yet humble.
Very neat explanation, explains the basics very well. Keep up the good work :)
@TheCodingTrain
7 жыл бұрын
Thank you!
Bro, this is so much fun to learn.. Thank you.. you explained it very well.. you really deserve a like and a subscribe
Great stuff as always! I appreciate you covering this statistical content :)
@TheCodingTrain
7 жыл бұрын
Glad to hear!
You are the best teacher I've ever had. Seriously. And I've had a LOT of them.
@TheCodingTrain
7 жыл бұрын
So nice, thank you!
I wish you had a teacher like yourself. If I were to compare this video to the entirety of what I learnt from my "programming" classes, I'd say I have wasted too many hours in said classes.
You're awesome! I love your videos, they're always really interesting ! Big up from France :p
@TheCodingTrain
7 жыл бұрын
Merci!
Thanks for the video. your demeanor really brightened my day.
Thank you AP Stat. I actually understood all of this
Great explanation! Super straightforward! 10 / 10 !
@TheCodingTrain
7 жыл бұрын
Thank you!
6 years later and this content is helping me out
Awesome video! Entertaining. Keep up the awesome job =)
Wow. You're awesome. I was able to understand linear regression with your video.
Amazing video - Your videos are the best dumb (dumb being me) it down versions I have found in ML. Thank you very much
fun learning with you always :)
Amazing presentation, Thanks!
Thank you so much,this lecture has become really helpful to me
My man, you're a beast!
You're an awesome teacher!
AMAZINGGG So helpful sir .Thank you sooooo much
I just ❤ love your energy!
This channel is a gem
@TheCodingTrain
7 жыл бұрын
Thanks for watching!
awesome video. thank you for sharing your knowledge was well as your jokes with us. you are GREAT
What a freakin great video :D Thank you!
After googling for what I needed to know to do my project I have ended up on this video and channel learning about OLS. Although it did help me to understand better what is OLS I do't think I can do this on my own. It would be cool if anyone from the comment section or Daniel could help me out with my problem. So, I'm trying to create a math solution that would give me soccer players post game ratings based on the actions done during the game. It seems that googling is keeping me in a loop around Linear Regression, as it seems that such math model has been used, in combination with some others, to do a real life predictions and post game rating for soccer players by various online sites and soccer journals. The system I would like to create, first and foremost as a math model and then later as an actual code, would be fed by 30+ events (goals, assists, block, saves, etc) that should quantify all of that as a number on a scale from 1 to 10 with 2 decimal points as a precision. Is OLS the right way to go? Perhaps other Linear Regression model? Maybe totally different math model and a formula? Where do I event start when I do pick a model? Decide how much of these actions are worth and then fit them in the model? Any help would be great. Thanks.
wow you are great man keep going like this :)
awesome I needed this
I reallly enjoy your videos, you are really amazing
very nicely explain :)
u make math interesting for ur student..good job sir
I wish I had teacher like this.
It was very helpful
nice video..liked it a lot
Presumably you either want to go from i = 1 to n in x_bar, or divide by (n+1), since there are n+1 terms in that sum (from 0 to n)
@TheCodingTrain
7 жыл бұрын
Yes, this was a mistake, thank you!
@11:17 "...all of the X-files[viles?], values..." lol Just shows how your brain is always making connections everywhere! Great videos! Your stuff always makes my day :)
@lelsewherelelsewhere9435
3 жыл бұрын
Also as another commenter said, "just to recap for a second" while recapping the marker 14:22. Making connections everywhere!
Thanks for this video, you are really good
Oh. Oh wow, I never saw it when I actually studied stats but it looks like this is where the i=0 to n loop counter convention comes from. If the relationship isn't that direct it's still very similar and helps my understanding.
"I head there's this channel, with tutorials and a whiteboard, where they use multiple colours". Check out blackpenredpen.
Sir your videos are amaizing.
@TheCodingTrain
7 жыл бұрын
Thank you!
I already said this, but this man is gold
Thank you so much you saved me
Thanks. Keep it up.
With this level of hotness he can teach me anything 🔥
I would love to have you as my lecturer for all my units. You have a gift in teaching
not sure i understand how the final slope formula (m) arrived. formula for a slope is rise/run i.e. (y2-y1)/(x2-x1). how did we get the value of slope as the final formula shown as summation
@anjaangyanshots1310
3 жыл бұрын
Did you get that?? Please explain to me too
U are a great teacher ❤😂
Needs more "this."
Looking forward for next videos on this subject!
@TheCodingTrain
7 жыл бұрын
Thanks for watching!
In the uk our general equation of a line is y=mx+c. Whichever country changed it is just petty 😂
@gioni_mango
7 жыл бұрын
Lewis Crosby Belgium here, we use y=mx+p or ax+b
@guygeva7375
7 жыл бұрын
third degree: y=ax^3+bx^2+cx+d second degree: y=ax^2+bx+c first degree: y=mx+b *sigh* You probably have it as c because they wanted it to be similiar to second degree formulas or something. who knows :P
@sonicsalies
7 жыл бұрын
Brazil: ax + b
@stn4873
6 жыл бұрын
in my room y= z! + v?
@ThePlayerNotJump1
6 жыл бұрын
India uses y=mx+c as well... Pretty sure we all know why... Thanks English people :D
there's already a channel on KZread that is called "blackpenredpen" and he uses black and red markers to explain math
it gives degree of freedom to move the line-up and down when plotting on the line using data points
If you calculate the sum (for x or for m), shouldnt it always be from 1 to n instead of 0 to n?
Your videos are great❣ I'm new to ML Can you please rearrange the series for ML so that it is not messy please
Intead of using square values of difference between the predicted value and observed value why dont we take absolute value of their difference?Please tell
If n is the total, shouldn't i run from 0 to n-1 ? Otherwise, great videos and attitude ! Subbed.
why does not the fitted linear regression line go through the origin?? why does it intercept Y axis??
Squaring the values also has a different effect. If the line is somewhere between two points and you don't square, it doesn't care where the line is, the distance is always the sum of the two points. If you take the square though it changes that. Let's say there are two point 10 away from each other. The line could be directly on one of these points, the total distance is still 10 to the line from both points but the sum of the quare distance is 100. If the line is directly in the middle you have two times a distance of 5. Again just sum of distance is 10 but the square distance suddenly just ist 2 * (5 ^ 2) which is 50. That's why squaring it makes sense. 6:50
@TheCodingTrain
7 жыл бұрын
Thanks for this excellent clarification!
I need to relate the data points to the best fit line in terms of land surveying where the data points are perpendicular to the best fit line. This is true for land surveying where the data points represent property corners and would be left or right of the best fit line. Your video shows the data points directly south (Y-axis) of the best fit line. How do I relate your example to my needs?
I heard this guy used multiple colours. Subscribed.
Could you give a link to the proof of the slope formula ?
Hey! I'm looking for some fun challenges in javascript, do you have any challenges you'd recommend? p.s. I watched the live stream in which you made this video, and you mentioned Tetris. I made it in like a day, so you probably can in one video :P Would be cool to see you make it!
@Blazeorite
7 жыл бұрын
Kaixo Music is java script the most fun language?
@Kaixo
7 жыл бұрын
Idk, It's the second (third if you count HTML) language I learned... the first one was C. So I don't really know...
his vibes are off the charts
I ressubed for the multicolored markers. Defenatly didn't unsub earlier just to do this. Most defenatly not.
@TheCodingTrain
7 жыл бұрын
best comment
perfect!
So where can I bid on the first preparation paper? :D
Could you please make a coding challenge where you code a cylindrical coordinate simulator aplet? The aplet should be in 3d and have sliders for adjusting the radius, angle and height. It should be entertaining and cool to program
Awesome
i have an idea : sometimes you write off the board so to make sure that you don't you could just draw a line to mark the limits of your draw-maths zone ?! (it would ony work if you place your camera always at the same place) =p PS : i love what you do =) please carry on !!! =D
@TheCodingTrain
7 жыл бұрын
A very good suggestion indeed!
I become the fan of your way of teaching :) Do you have any online courses on Udemy?
But how do you know that its not a quadratic relationship or a cubic one etc.
You should try an LSTM for regression, since they are good for time series data
@TheCodingTrain
7 жыл бұрын
Thanks Jyo!
Great
sir what are perquisites for machine learning course , I know c++ and data structure only
@TheCodingTrain
7 жыл бұрын
You should be good here, I assume very little!
Epic dab at 3:50 😂😂
If the difference is positive and negative then wouldn't adding them up minimize the residual value? Why square them?
Love it 😂 04:17
OoooKKKK, I got a sudden shocked by the opening casual kiddy music...
a very interesting guy
This guy is legit bruh
Yay!
@omollojude6337
7 жыл бұрын
I love the presentation, as always
Great teacher, everything is the best...Will you makr anything in C++ because you said in livestream you know it. But, can you makr an OO C++ Serial? I am really good in C++, but I can't understand classes, because I don't know why not to write single void, instead to make a full class. Can you please do something in C++?
@xzencombo3400
7 жыл бұрын
Btw. I am first :)
@TheCodingTrain
7 жыл бұрын
I've got a long list of things to get to, but I'm not opposed to doing C++ at some point. Maybe I can get some guest tutorials on this topic.
@xzencombo3400
7 жыл бұрын
Ahh, ok then...Btw. ty for replying :)
@dhritimanbose2186
7 жыл бұрын
xZen combo
nice