3.2: Linear Regression with Ordinary Least Squares Part 1 - Intelligence and Learning

In this video, part of my series on "Machine Learning", I explain how to perform Linear Regression for a 2D dataset using the Ordinary Least Squares method. In Part 2, I demonstrate how to code the algorithm in JavaScript, using the p5.js library
This video is part of session 3 of my Spring 2017 ITP "Intelligence and Learning" course (github.com/shiffman/NOC-S17-2...)
Link to Part 2: • 3.3: Linear Regression...
Support this channel on Patreon: / codingtrain
To buy Coding Train merchandise: www.designbyhumans.com/shop/c...
Send me your questions and coding challenges!: github.com/CodingTrain/Rainbo...
Contact:
Twitter: / shiffman
The Coding Train website: thecodingtrain.com/
Links discussed in this video:
Session 3 of Intelligence and Learning: github.com/shiffman/NOC-S17-2...
Nature of Code: natureofcode.com/
kwichmann's Linear Regression Diagnostics: kwichmann.github.io/ml_sandbo...
Linear Regression on Wikipedia: en.wikipedia.org/wiki/Linear_...
Anscombe's quartet on Wikipedia: en.wikipedia.org/wiki/Anscomb...
Source Code for the all Video Lessons: github.com/CodingTrain/Rainbo...
p5.js: p5js.org/
Processing: processing.org
For More Coding Challenges: • Coding Challenges
For More Intelligence and Learning: • Intelligence and Learning
Help us caption & translate this video!
amara.org/v/7UbL/
📄 Code of Conduct: github.com/CodingTrain/Code-o...

Пікірлер: 220

  • @RodamisCZ
    @RodamisCZ7 жыл бұрын

    I have linear depression

  • @112BALAGE112

    @112BALAGE112

    7 жыл бұрын

    Happiness goes straight down.

  • @extradimension7356

    @extradimension7356

    7 жыл бұрын

    If you don't find help it can be iterative problem

  • @kayinb90

    @kayinb90

    4 жыл бұрын

    😂 me too lmfao

  • @MrBollec

    @MrBollec

    4 жыл бұрын

    I hope that slope was negative ;)

  • @adamlanghans
    @adamlanghans7 жыл бұрын

    Subbed for multiple color markers.

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Mission accomplished!

  • @csanadtemesvari9251

    @csanadtemesvari9251

    5 жыл бұрын

    Check out blackpenredpen then!

  • @sii479

    @sii479

    4 жыл бұрын

    Same

  • @whatsup7341
    @whatsup73417 жыл бұрын

    "I'm really sorry, I'm going to do this in Fahrenheit, I have to apologize for that. I'll happily say "maths" though." You crack me up Dan

  • @JFrameMan
    @JFrameMan4 жыл бұрын

    Great explanation. You're the only channel that has such a nice balance between explaining academic content and programming it.

  • @wookotech4216
    @wookotech42165 жыл бұрын

    I am smiling now...I was very confused till i watched your explanation. Thank you Sir!

  • @sidindian1982
    @sidindian1982 Жыл бұрын

    ordinary least Square .. Very well explained Sir :-) ... i am preparing for data science as my carrier ...

  • @Oompie279
    @Oompie2797 жыл бұрын

    Nice video! The numerator of the expression for the slope is actually the sample covariance of X and Y. Sample covariance is a measure of linear dependence. The denominator is the sample variance of X. So the slope is loosely said the linear dependence of X and Y, normalized by the variability in X. Hope that makes sense!

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Yes, it does, thank you!!

  • @clabe45
    @clabe457 жыл бұрын

    Nice job re-capping the marker as you said "recap" at 14:22.

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Hah!

  • @hamad.learns

    @hamad.learns

    5 жыл бұрын

    Dude how do you even recognize that !

  • @im_karamo1907
    @im_karamo19076 жыл бұрын

    heehhhe.. bro honestly you have win my day. your sense of humor makes me understand it greatly. congrats again. hehe

  • @jude_ghenon
    @jude_ghenon4 жыл бұрын

    Great job! This is the nth video of yours I've watched and you've been able to explain hard concepts like AI algorithms like they were nuts! It seems they're so easy and I don't know if you're too good to explain or I'm so dumb I'm not catching the things right.

  • @rapi1975
    @rapi19756 жыл бұрын

    Thanks for the video. finally easy explanation of linear regression.

  • @trickyabb
    @trickyabb6 жыл бұрын

    It all makes sense now thank you for saving the day and you make learning fun :)

  • @paulkerrigan9857
    @paulkerrigan98573 жыл бұрын

    You’re one of my heroes, man. Seeing you make these coding videos years ago inspired me to learn new languages and technologies. Playing with P5.js is lots of fun.

  • @HarshTalpada
    @HarshTalpada7 жыл бұрын

    Dan, you are precisely the one, that our education institutions currently need, simply amazing way of making anyone understand even the most complex stuff

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thank you so much!

  • @thevinaymittal.
    @thevinaymittal.6 жыл бұрын

    Great work sir, You have an art of teaching with fun.

  • @adilkhatri7475
    @adilkhatri74756 жыл бұрын

    one of the best explanation on linear regression! thank you!

  • @kaartiikvijayan1623
    @kaartiikvijayan16235 жыл бұрын

    I was having trouble understanding this. Thanks to you I get it now!!

  • @synju
    @synju7 жыл бұрын

    Please BIG Please, continue with this series. I've followed along with your genetic algorithms, and now I'm busy with neural networks, so please continue with this. Would really appreciate it.

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thanks, I am going to!

  • @h.sharathkumar1889
    @h.sharathkumar18895 жыл бұрын

    You are just awesome. ..so intelligent yet humble.

  • @jadlteif8143
    @jadlteif81437 жыл бұрын

    Very neat explanation, explains the basics very well. Keep up the good work :)

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thank you!

  • @omkarfadtare3054
    @omkarfadtare30542 жыл бұрын

    Bro, this is so much fun to learn.. Thank you.. you explained it very well.. you really deserve a like and a subscribe

  • @Boop57323
    @Boop573237 жыл бұрын

    Great stuff as always! I appreciate you covering this statistical content :)

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Glad to hear!

  • @lucasmontec
    @lucasmontec7 жыл бұрын

    You are the best teacher I've ever had. Seriously. And I've had a LOT of them.

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    So nice, thank you!

  • @borismilenski4759
    @borismilenski47597 жыл бұрын

    I wish you had a teacher like yourself. If I were to compare this video to the entirety of what I learnt from my "programming" classes, I'd say I have wasted too many hours in said classes.

  • @MadSandman
    @MadSandman7 жыл бұрын

    You're awesome! I love your videos, they're always really interesting ! Big up from France :p

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Merci!

  • @jaykfrisby8163
    @jaykfrisby81633 жыл бұрын

    Thanks for the video. your demeanor really brightened my day.

  • @neilbolima9694
    @neilbolima96946 жыл бұрын

    Thank you AP Stat. I actually understood all of this

  • @DamianReloaded
    @DamianReloaded7 жыл бұрын

    Great explanation! Super straightforward! 10 / 10 !

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thank you!

  • @JoseLopez-oz5tn
    @JoseLopez-oz5tn Жыл бұрын

    6 years later and this content is helping me out

  • @gibsonrobb
    @gibsonrobb4 жыл бұрын

    Awesome video! Entertaining. Keep up the awesome job =)

  • @robertop.556
    @robertop.556 Жыл бұрын

    Wow. You're awesome. I was able to understand linear regression with your video.

  • @shankarkarthik
    @shankarkarthik6 жыл бұрын

    Amazing video - Your videos are the best dumb (dumb being me) it down versions I have found in ML. Thank you very much

  • @harshitsharma589
    @harshitsharma5893 жыл бұрын

    fun learning with you always :)

  • @krzysztofkuliga2148
    @krzysztofkuliga21486 жыл бұрын

    Amazing presentation, Thanks!

  • @angelrai4897
    @angelrai48974 жыл бұрын

    Thank you so much,this lecture has become really helpful to me

  • @flader_lol
    @flader_lol4 жыл бұрын

    My man, you're a beast!

  • @willykitheka7618
    @willykitheka76184 жыл бұрын

    You're an awesome teacher!

  • @stancydube5268
    @stancydube52686 жыл бұрын

    AMAZINGGG So helpful sir .Thank you sooooo much

  • @willykitheka7618
    @willykitheka76184 жыл бұрын

    I just ❤ love your energy!

  • @Shaffan93
    @Shaffan937 жыл бұрын

    This channel is a gem

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thanks for watching!

  • @mohabibrahime7397
    @mohabibrahime73974 жыл бұрын

    awesome video. thank you for sharing your knowledge was well as your jokes with us. you are GREAT

  • @ThelarionMusic
    @ThelarionMusic4 жыл бұрын

    What a freakin great video :D Thank you!

  • @zexivic7740
    @zexivic77404 жыл бұрын

    After googling for what I needed to know to do my project I have ended up on this video and channel learning about OLS. Although it did help me to understand better what is OLS I do't think I can do this on my own. It would be cool if anyone from the comment section or Daniel could help me out with my problem. So, I'm trying to create a math solution that would give me soccer players post game ratings based on the actions done during the game. It seems that googling is keeping me in a loop around Linear Regression, as it seems that such math model has been used, in combination with some others, to do a real life predictions and post game rating for soccer players by various online sites and soccer journals. The system I would like to create, first and foremost as a math model and then later as an actual code, would be fed by 30+ events (goals, assists, block, saves, etc) that should quantify all of that as a number on a scale from 1 to 10 with 2 decimal points as a precision. Is OLS the right way to go? Perhaps other Linear Regression model? Maybe totally different math model and a formula? Where do I event start when I do pick a model? Decide how much of these actions are worth and then fit them in the model? Any help would be great. Thanks.

  • @kerolesmonsef4179
    @kerolesmonsef41796 жыл бұрын

    wow you are great man keep going like this :)

  • @miliano777
    @miliano777 Жыл бұрын

    awesome I needed this

  • @christianfernandez7385
    @christianfernandez73855 жыл бұрын

    I reallly enjoy your videos, you are really amazing

  • @crazyjpr2009
    @crazyjpr20096 жыл бұрын

    very nicely explain :)

  • @agusnoviana
    @agusnoviana Жыл бұрын

    u make math interesting for ur student..good job sir

  • @amranmohamed377
    @amranmohamed3772 жыл бұрын

    I wish I had teacher like this.

  • @sajithmarlon9489
    @sajithmarlon94893 жыл бұрын

    It was very helpful

  • @vijaykumarkari
    @vijaykumarkari6 жыл бұрын

    nice video..liked it a lot

  • @gigelfranaru
    @gigelfranaru7 жыл бұрын

    Presumably you either want to go from i = 1 to n in x_bar, or divide by (n+1), since there are n+1 terms in that sum (from 0 to n)

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Yes, this was a mistake, thank you!

  • @lelsewherelelsewhere9435
    @lelsewherelelsewhere94353 жыл бұрын

    @11:17 "...all of the X-files[viles?], values..." lol Just shows how your brain is always making connections everywhere! Great videos! Your stuff always makes my day :)

  • @lelsewherelelsewhere9435

    @lelsewherelelsewhere9435

    3 жыл бұрын

    Also as another commenter said, "just to recap for a second" while recapping the marker 14:22. Making connections everywhere!

  • @Pylsng
    @Pylsng3 жыл бұрын

    Thanks for this video, you are really good

  • @tenou213
    @tenou2136 жыл бұрын

    Oh. Oh wow, I never saw it when I actually studied stats but it looks like this is where the i=0 to n loop counter convention comes from. If the relationship isn't that direct it's still very similar and helps my understanding.

  • @midwinter78
    @midwinter787 жыл бұрын

    "I head there's this channel, with tutorials and a whiteboard, where they use multiple colours". Check out blackpenredpen.

  • @ordinarycoder8090
    @ordinarycoder80907 жыл бұрын

    Sir your videos are amaizing.

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thank you!

  • @climito
    @climito Жыл бұрын

    I already said this, but this man is gold

  • @wise_394
    @wise_394 Жыл бұрын

    Thank you so much you saved me

  • @liminal18
    @liminal187 жыл бұрын

    Thanks. Keep it up.

  • @ankitasaxena5891
    @ankitasaxena5891 Жыл бұрын

    With this level of hotness he can teach me anything 🔥

  • @truthseer7904
    @truthseer79045 жыл бұрын

    I would love to have you as my lecturer for all my units. You have a gift in teaching

  • @somakkamos
    @somakkamos5 жыл бұрын

    not sure i understand how the final slope formula (m) arrived. formula for a slope is rise/run i.e. (y2-y1)/(x2-x1). how did we get the value of slope as the final formula shown as summation

  • @anjaangyanshots1310

    @anjaangyanshots1310

    3 жыл бұрын

    Did you get that?? Please explain to me too

  • @pauldominic2964
    @pauldominic29645 ай бұрын

    U are a great teacher ❤😂

  • @Remls
    @Remls7 жыл бұрын

    Needs more "this."

  • @practicalsoftwaremarcus
    @practicalsoftwaremarcus7 жыл бұрын

    Looking forward for next videos on this subject!

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thanks for watching!

  • @lewiscrosby2074
    @lewiscrosby20747 жыл бұрын

    In the uk our general equation of a line is y=mx+c. Whichever country changed it is just petty 😂

  • @gioni_mango

    @gioni_mango

    7 жыл бұрын

    Lewis Crosby Belgium here, we use y=mx+p or ax+b

  • @guygeva7375

    @guygeva7375

    7 жыл бұрын

    third degree: y=ax^3+bx^2+cx+d second degree: y=ax^2+bx+c first degree: y=mx+b *sigh* You probably have it as c because they wanted it to be similiar to second degree formulas or something. who knows :P

  • @sonicsalies

    @sonicsalies

    7 жыл бұрын

    Brazil: ax + b

  • @stn4873

    @stn4873

    6 жыл бұрын

    in my room y= z! + v?

  • @ThePlayerNotJump1

    @ThePlayerNotJump1

    6 жыл бұрын

    India uses y=mx+c as well... Pretty sure we all know why... Thanks English people :D

  • @jfr9964
    @jfr99645 жыл бұрын

    there's already a channel on KZread that is called "blackpenredpen" and he uses black and red markers to explain math

  • @datasciencegyan5145
    @datasciencegyan51452 жыл бұрын

    it gives degree of freedom to move the line-up and down when plotting on the line using data points

  • @fabianstutzle9769
    @fabianstutzle97695 жыл бұрын

    If you calculate the sum (for x or for m), shouldnt it always be from 1 to n instead of 0 to n?

  • @sreeharsha1047
    @sreeharsha10474 жыл бұрын

    Your videos are great❣ I'm new to ML Can you please rearrange the series for ML so that it is not messy please

  • @ashutoshrai5342
    @ashutoshrai53424 жыл бұрын

    Intead of using square values of difference between the predicted value and observed value why dont we take absolute value of their difference?Please tell

  • @AlchemyOfDeathBand
    @AlchemyOfDeathBand7 жыл бұрын

    If n is the total, shouldn't i run from 0 to n-1 ? Otherwise, great videos and attitude ! Subbed.

  • @maisarahrashid8478
    @maisarahrashid84784 жыл бұрын

    why does not the fitted linear regression line go through the origin?? why does it intercept Y axis??

  • @TNGOMinecraft
    @TNGOMinecraft7 жыл бұрын

    Squaring the values also has a different effect. If the line is somewhere between two points and you don't square, it doesn't care where the line is, the distance is always the sum of the two points. If you take the square though it changes that. Let's say there are two point 10 away from each other. The line could be directly on one of these points, the total distance is still 10 to the line from both points but the sum of the quare distance is 100. If the line is directly in the middle you have two times a distance of 5. Again just sum of distance is 10 but the square distance suddenly just ist 2 * (5 ^ 2) which is 50. That's why squaring it makes sense. 6:50

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thanks for this excellent clarification!

  • @fielddog8111
    @fielddog81112 жыл бұрын

    I need to relate the data points to the best fit line in terms of land surveying where the data points are perpendicular to the best fit line. This is true for land surveying where the data points represent property corners and would be left or right of the best fit line. Your video shows the data points directly south (Y-axis) of the best fit line. How do I relate your example to my needs?

  • @ManAcadie
    @ManAcadie3 жыл бұрын

    I heard this guy used multiple colours. Subscribed.

  • @MrWabouz
    @MrWabouz6 жыл бұрын

    Could you give a link to the proof of the slope formula ?

  • @Kaixo
    @Kaixo7 жыл бұрын

    Hey! I'm looking for some fun challenges in javascript, do you have any challenges you'd recommend? p.s. I watched the live stream in which you made this video, and you mentioned Tetris. I made it in like a day, so you probably can in one video :P Would be cool to see you make it!

  • @Blazeorite

    @Blazeorite

    7 жыл бұрын

    Kaixo Music is java script the most fun language?

  • @Kaixo

    @Kaixo

    7 жыл бұрын

    Idk, It's the second (third if you count HTML) language I learned... the first one was C. So I don't really know...

  • @hollykm
    @hollykm2 жыл бұрын

    his vibes are off the charts

  • @guygeva7375
    @guygeva73757 жыл бұрын

    I ressubed for the multicolored markers. Defenatly didn't unsub earlier just to do this. Most defenatly not.

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    best comment

  • @ShahnewazKhan
    @ShahnewazKhan6 жыл бұрын

    perfect!

  • @polyclickkkkk
    @polyclickkkkk6 жыл бұрын

    So where can I bid on the first preparation paper? :D

  • @rafainfernal
    @rafainfernal7 жыл бұрын

    Could you please make a coding challenge where you code a cylindrical coordinate simulator aplet? The aplet should be in 3d and have sliders for adjusting the radius, angle and height. It should be entertaining and cool to program

  • @nagesh007
    @nagesh007 Жыл бұрын

    Awesome

  • @tjcraft7233
    @tjcraft72337 жыл бұрын

    i have an idea : sometimes you write off the board so to make sure that you don't you could just draw a line to mark the limits of your draw-maths zone ?! (it would ony work if you place your camera always at the same place) =p PS : i love what you do =) please carry on !!! =D

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    A very good suggestion indeed!

  • @prachikhandelwal2259
    @prachikhandelwal22594 жыл бұрын

    I become the fan of your way of teaching :) Do you have any online courses on Udemy?

  • @yakov9ify
    @yakov9ify7 жыл бұрын

    But how do you know that its not a quadratic relationship or a cubic one etc.

  • @JyoPari
    @JyoPari7 жыл бұрын

    You should try an LSTM for regression, since they are good for time series data

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    Thanks Jyo!

  • @thehappycoder3760
    @thehappycoder37602 жыл бұрын

    Great

  • @akshatBountyHunter
    @akshatBountyHunter7 жыл бұрын

    sir what are perquisites for machine learning course , I know c++ and data structure only

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    You should be good here, I assume very little!

  • @adwaitraste8904
    @adwaitraste89047 жыл бұрын

    Epic dab at 3:50 😂😂

  • @FerdousHasan-kk8hp
    @FerdousHasan-kk8hp2 жыл бұрын

    If the difference is positive and negative then wouldn't adding them up minimize the residual value? Why square them?

  • @huraibyel-huraiby7462
    @huraibyel-huraiby74622 жыл бұрын

    Love it 😂 04:17

  • @cheewanng6138
    @cheewanng61384 жыл бұрын

    OoooKKKK, I got a sudden shocked by the opening casual kiddy music...

  • @leilachaltu2314
    @leilachaltu231411 ай бұрын

    a very interesting guy

  • @eduardusadrian225
    @eduardusadrian2252 жыл бұрын

    This guy is legit bruh

  • @Goel25
    @Goel257 жыл бұрын

    Yay!

  • @omollojude6337

    @omollojude6337

    7 жыл бұрын

    I love the presentation, as always

  • @xzencombo3400
    @xzencombo34007 жыл бұрын

    Great teacher, everything is the best...Will you makr anything in C++ because you said in livestream you know it. But, can you makr an OO C++ Serial? I am really good in C++, but I can't understand classes, because I don't know why not to write single void, instead to make a full class. Can you please do something in C++?

  • @xzencombo3400

    @xzencombo3400

    7 жыл бұрын

    Btw. I am first :)

  • @TheCodingTrain

    @TheCodingTrain

    7 жыл бұрын

    I've got a long list of things to get to, but I'm not opposed to doing C++ at some point. Maybe I can get some guest tutorials on this topic.

  • @xzencombo3400

    @xzencombo3400

    7 жыл бұрын

    Ahh, ok then...Btw. ty for replying :)

  • @dhritimanbose2186

    @dhritimanbose2186

    7 жыл бұрын

    xZen combo

  • @suyogsudrik1820
    @suyogsudrik18206 жыл бұрын

    nice