Deriving the least squares estimators of the slope and intercept (simple linear regression)

I derive the least squares estimators of the slope and intercept in simple linear regression (Using summation notation, and no matrices.) I assume that the viewer has already been introduced to the linear regression model, but I do provide a brief review in the first few minutes. I assume that you have a basic knowledge of differential calculus, including the power rule and the chain rule.
If you are already familiar with the problem, and you are just looking for help with the mathematics of the derivation, the derivation starts at 3:26.
At the end of the video, I illustrate that sum(X_i-X bar)(Y_i - Y bar) = sum X_i(Y_i - Y bar) =sum Y_i(X_i - X bar) , and that sum(X_i-X bar)^2 = sum X_i(X_i - X bar).
There are, of course, a number of ways of expressing the formula for the slope estimator, and I make no attempt to list them all in this video.

Пікірлер: 194

  • @AnDr3s0
    @AnDr3s04 жыл бұрын

    Finally, someone who made it simple to understand! Thank you!

  • @victoriabrimm5014

    @victoriabrimm5014

    3 жыл бұрын

    Right! i went through like a million videos trying to understand this one segment and this was the first to do it.

  • @agustinlawtaro

    @agustinlawtaro

    3 жыл бұрын

    True.

  • @cmacompilation4649

    @cmacompilation4649

    6 ай бұрын

    please, how is it possible to consider Beta variable (when taking derivatives) and then consider Beta constant (to take it out of the sum) ???

  • @cleisonarmandomanriqueagui9176

    @cleisonarmandomanriqueagui9176

    Ай бұрын

    Best video . i was looking for something like this

  • @anuraghavsai5575
    @anuraghavsai55753 жыл бұрын

    This is an underrated video.

  • @divitisaitezaa2599

    @divitisaitezaa2599

    3 жыл бұрын

    @Isaiah Matias is it?

  • @sajibmannan
    @sajibmannan3 жыл бұрын

    My god, you explained this so easily. It took me hours trying to understand this before watching this video but still couldn’t understand it properly. After watching this video, it's crystal-clear now. ❤️

  • @DHDH_DH

    @DHDH_DH

    5 ай бұрын

    me, too. I spent a whole morning figuring this. He is a savior

  • @user-jk2tp6gs2g
    @user-jk2tp6gs2g2 жыл бұрын

    I have gone through tons of materials on this topic and they either skip the derivation process or go direct into some esoteric matrix arithmetics. This video explains everything I need to know. Thanks.

  • @alimortadahoumani1134
    @alimortadahoumani11347 ай бұрын

    unbelievably perfect video, one of the best videos I have watched in the statistics field, so rare to find high-quality in this field idk why

  • @augustinejunior3361
    @augustinejunior33612 жыл бұрын

    I never thought that I could understand simple linear regression using this approach. Thank you

  • @amberxv4777
    @amberxv47772 жыл бұрын

    I don't usually comment on teaching videos. But this really deserves thanks for how clearly and simply you explained everything. The lecture I had at the university left much to be desired

  • @aaskyboi
    @aaskyboi4 жыл бұрын

    I can't thank you enough for this brilliant explanation!

  • @JFstriderXgaming
    @JFstriderXgaming3 жыл бұрын

    My Physical Chemistry teacher spent ~1.5 hrs showing this derivation and I got completely lost. Watching your video, it's so clear now. Thank you for your phenomenal work.

  • @monojitchatterjee3185
    @monojitchatterjee31854 жыл бұрын

    Absolutely beautiful derivation! Crystal clear! Thanks very much.

  • @mns003-choudharyvivek2

    @mns003-choudharyvivek2

    9 ай бұрын

    Can I apply this in Mumbai University exam?

  • @nak6608
    @nak66082 жыл бұрын

    phenomenal video. Thank you for taking the time to explain each step of the derivations such as the sum rule for derivation. Thank you for helping me learn.

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    You are very welcome!

  • @yixuanliu8368
    @yixuanliu8368 Жыл бұрын

    you have no idea how you saved my life, I was struggling so hard to find out why xi(xi-xbar)=(xi-xbar)^2 and etc. you are the first one I found explained that.

  • @pkeric2626
    @pkeric26264 жыл бұрын

    Thanks so much, this was so easy to follow and comprehend!

  • @Murraythis
    @Murraythis2 жыл бұрын

    Amazing video! Slight bumps where my own knowledge was patchy but you provided enough steps for me to work those gaps out.

  • @valeriereid2337
    @valeriereid2337 Жыл бұрын

    The best part of this video is finally figuring out where that "n" came from in the equation for beta-naught-hat. Thank you so very much for making this available.

  • @jbstatistics

    @jbstatistics

    Жыл бұрын

    I'm glad to be of help!

  • @ghunter958
    @ghunter9584 жыл бұрын

    Really, really good explanation!! Thank you!!

  • @TheProblembaer2
    @TheProblembaer2 Жыл бұрын

    This is FANTASTIC. THANK YOU!

  • @danverzhao9912
    @danverzhao99122 жыл бұрын

    Thank you so much! This explanation is literally perfect, helped me so much!

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    Thanks for the kind words! I'm glad to be of help!

  • @jingyiwang5113
    @jingyiwang51137 ай бұрын

    Thank you so much for such a clear explanation! It helps me a lot in preparing for my upcoming final exam.

  • @martijnbos9873
    @martijnbos98734 жыл бұрын

    Absolutely brilliant video!!! Thanks so much

  • @muhammaddzakyrafliansyah3587
    @muhammaddzakyrafliansyah3587 Жыл бұрын

    You made it simpler than my lecturer do. Thank you!

  • @asad_rez
    @asad_rez2 жыл бұрын

    Thank you for explaining in such details ❤️

  • @tastypie2276
    @tastypie22763 жыл бұрын

    Thank you very much! Very clear and interesting explanation!

  • @daniyal98
    @daniyal984 жыл бұрын

    one video on youtube that actually explains something properly

  • @process6996
    @process69965 жыл бұрын

    Glad you're back!

  • @jbstatistics

    @jbstatistics

    5 жыл бұрын

    Thanks! Glad to be back! Just recording and editing as I type this!

  • @eltuneyvazzade8845
    @eltuneyvazzade8845 Жыл бұрын

    Very helpful video to understand. Many thanks!

  • @jackhasfun4752
    @jackhasfun47528 ай бұрын

    thank you so much, this video has cleared all my confusions cuz the book im reading just says 'by doing some simple calculus'

  • @satyarath7723
    @satyarath77232 жыл бұрын

    Thanks a lot sir I really got this what I need indeed. 🙏🙏🙏🙏🙏🙏🙏🙏There is no words for appreciation of your efforts

  • @TheMatthyssen
    @TheMatthyssen7 ай бұрын

    thank you for actually explaining it, most of videos are just like "hi, if you want to solve this, plug in this awesome formula and thats it, thank you for watching :)"

  • @aron4317
    @aron43172 ай бұрын

    Beautiful video, good explanation

  • @raitup00
    @raitup004 жыл бұрын

    Great explanation! Step by step...

  • @Zydres_Impaler
    @Zydres_Impaler Жыл бұрын

    thanks a lot for simplifying the derivation

  • @wuttinanlerkmangkorn7009
    @wuttinanlerkmangkorn70092 жыл бұрын

    Best explanation, thank you so much

  • @preethidonthu2433
    @preethidonthu2433 Жыл бұрын

    Thank u soooo much! For explaining this. You made my day

  • @ericricky808
    @ericricky8083 жыл бұрын

    This is talent. Thank you so much 😊

  • @AJ-et3vf
    @AJ-et3vf2 жыл бұрын

    Awesome video sir! Thank you!

  • @olympusexothermic1752
    @olympusexothermic1752 Жыл бұрын

    Thank you so much am really enjoying and understanding what your teaching

  • @loden5677
    @loden56777 ай бұрын

    This was really helpful thanks!

  • @mangaart3366
    @mangaart33663 жыл бұрын

    Exactly what I was looking for. Thank you so much!

  • @jbstatistics

    @jbstatistics

    3 жыл бұрын

    You are very welcome!

  • @herodmoonga4799

    @herodmoonga4799

    2 жыл бұрын

    Greatful, you are wonderful Sir,you have made me understand economics

  • @fatemehmaroufkhani7104
    @fatemehmaroufkhani71042 жыл бұрын

    Thank you very much. This video helped me a lot.

  • @frequencyspectra1
    @frequencyspectra12 жыл бұрын

    Excellent video

  • @soryegetun529
    @soryegetun5292 жыл бұрын

    finally, I've understood this bloody thing. Thank u sooooo much m8.

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    I'm glad you found it helpful!

  • @louism.4980
    @louism.49805 ай бұрын

    This is incredible, thank you so much! :)

  • @jbstatistics

    @jbstatistics

    5 ай бұрын

    You are very welcome!

  • @mpandesyambayi5898
    @mpandesyambayi58982 жыл бұрын

    Greatly explained

  • @arjoy4942
    @arjoy49425 жыл бұрын

    Great sir, very helpful!

  • @CaptainCalculus
    @CaptainCalculus3 жыл бұрын

    Excellent video!

  • @DHDH_DH
    @DHDH_DH5 ай бұрын

    You are awesome! I am not a native speaker and still struggling with the master program courses in the US, but your instruction is so helpful. I appreciate your great help

  • @jbstatistics

    @jbstatistics

    5 ай бұрын

    Thanks! I'm happy to be of help!

  • @renata8938
    @renata89383 жыл бұрын

    Thank you. This is very clear

  • @stretch8390
    @stretch8390 Жыл бұрын

    Easiest subscribe of my life.

  • @joansome951
    @joansome9519 ай бұрын

    Great job! Thank you sir!

  • @mohe4ever514
    @mohe4ever5142 жыл бұрын

    Very well explained

  • @promasterliuss
    @promasterliuss Жыл бұрын

    good explanation!

  • @girlstrends8204
    @girlstrends82045 ай бұрын

    you make it sooo easy

  • @tnorton314
    @tnorton3143 жыл бұрын

    Thanks, so helpful!

  • @poojachhachhiya1574
    @poojachhachhiya15743 жыл бұрын

    Wow. This is great.thanku so much.

  • @umedina98
    @umedina982 жыл бұрын

    Really nice derivation!

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    Thanks!

  • @jesabumaras4758
    @jesabumaras47582 жыл бұрын

    Thank you very to clear explanation ❤

  • @fisher4651
    @fisher46512 жыл бұрын

    LEGEND, HAVE TO SAY YOU ARE BETTER THAN A PROFFESOR

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    I *am* a professor!!!

  • @mautasimsiddiqui8618
    @mautasimsiddiqui8618 Жыл бұрын

    Thank you so much.

  • @sajibmannan
    @sajibmannan3 жыл бұрын

    Question: 6:24 Why and how beta zero hat is multiplied with n? Does n mean sample size? What's the reasoning behind n adjoining with beta zero hat?

  • @dalkeiththomas9352
    @dalkeiththomas93522 жыл бұрын

    You sir are AMazing

  • @omarzodiac
    @omarzodiac5 жыл бұрын

    Yes! New stuff 👍🏼👍🏼

  • @robin-bird
    @robin-bird2 жыл бұрын

    definitely the best video out there on this topic makes me wonder why its not the top recommendation / search result maybe because of the title

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    Thanks! How about "Finding the formulas for the slope and intercept the EASY WAY! (When I got to step 8, my jaw DROPPED!)" :)

  • @robin-bird

    @robin-bird

    2 жыл бұрын

    @@jbstatistics no idea, I'm not good at making up titles. Maybe something like this? Simple Linear Regression | Derivation

  • @jbstatistics

    @jbstatistics

    2 жыл бұрын

    @@robin-bird I was just joking, but thanks for the input :)

  • @robin-bird

    @robin-bird

    2 жыл бұрын

    @@jbstatistics I was wondering - thanks for clearing that up ^^

  • @LucyMburu605
    @LucyMburu60520 күн бұрын

    Thanks alot it really helped

  • @tezike
    @tezike3 жыл бұрын

    holy hell I wish you were my econometrics professor. mine is useless

  • @atomu9663
    @atomu9663 Жыл бұрын

    Good video Thanks!

  • @hellothere9298
    @hellothere92985 ай бұрын

    Thank you very much sir !

  • @jbstatistics

    @jbstatistics

    5 ай бұрын

    You are very welcome!

  • @ezozakamolova885
    @ezozakamolova8858 ай бұрын

    Thank you so much

  • @MudahnyaFizik
    @MudahnyaFizik3 жыл бұрын

    Thanks so much!

  • @sasirekhamsvl9504
    @sasirekhamsvl95043 жыл бұрын

    Great video Brother

  • @ethanchung1541
    @ethanchung15413 жыл бұрын

    u are a life saver

  • @zumichetiapator
    @zumichetiapatorАй бұрын

    finally got my doubt resolved.😊

  • @anangelsdiaries
    @anangelsdiaries2 ай бұрын

    The result represent the minimas since the original function that we were minimizing is convex and open upwards, so the only way for a critical value to exist is for it to be a minimum.

  • @katerinagk2681
    @katerinagk2681 Жыл бұрын

    THANK YOU

  • @user-fn6lz8pp6j
    @user-fn6lz8pp6j4 жыл бұрын

    THANKS GOD FINALLY SOMEONE TRIED TO DERIVE THE FORMULA, INSANE THAT NEARLY ALL OTHER RESOURCES OMIT THIS SHIT

  • @singsongsou1865
    @singsongsou18653 жыл бұрын

    Thanks a lot!!!

  • @IvanJacobsvelocity
    @IvanJacobsvelocity2 жыл бұрын

    thank you 100^100 times

  • @hawasadiki5292
    @hawasadiki52923 ай бұрын

    Thank you

  • @Harchit23
    @Harchit234 жыл бұрын

    this is great

  • @deborahsakazhila4068
    @deborahsakazhila40684 жыл бұрын

    Hi, do you have a video on deriving coefficients in multiple regression?

  • @mattstats399

    @mattstats399

    Жыл бұрын

    That is a fun derivation using linear algebra and calculus. First step is the same here which is taking the first derivative and setting it equal to zero. The book "The Elements of Statistical Learning" has a good proof. I'd say one needs a calc 1 and linear algebra background first though.

  • @MahnoorNaveed-
    @MahnoorNaveed-2 жыл бұрын

    thank u so much.

  • @ciasuryani5422
    @ciasuryani54224 жыл бұрын

    Thanks so much

  • @anannaanu9918
    @anannaanu99182 жыл бұрын

    Thanks a lot

  • @SiddharthPrabhu1983
    @SiddharthPrabhu19835 жыл бұрын

    Thanks Dr. Balka! Is it computationally expensive to estimate the parameters in this manner for models with many independent variables or very large datasets? Is that the reason why iterative methods such as gradient descent are sometimes employed instead?

  • @Tusharchitrakar

    @Tusharchitrakar

    5 ай бұрын

    The matrix inversion operation in ols is computationally expensive, hence numerical methods like gradient descent are useful.

  • @virgilhawkins4575
    @virgilhawkins45754 жыл бұрын

    Awesome!

  • @badrftouh9322
    @badrftouh9322 Жыл бұрын

    Nailed it

  • @AnuarPhysics
    @AnuarPhysics2 жыл бұрын

    Nice trick! Adding an intelligent zero huh? Thanks for this video!

  • @Sulegibongtoo176
    @Sulegibongtoo1762 жыл бұрын

    Wow many university lecturers can’t explain it this well!

  • @menghongpor2667
    @menghongpor26674 жыл бұрын

    At 10:52 timeline, how can we switch the role of X sub i and Y sub i? Could you help explain how this happens?

  • @harveywilliams7013

    @harveywilliams7013

    3 жыл бұрын

    In the first step, we choose to expand (Xi - Xbar) but we could have chosen to expand (Yi - Ybar) and it would follow a similar route.

  • @gan_luo1909
    @gan_luo19092 жыл бұрын

    saved my life

  • @xavieribanez-padron4857
    @xavieribanez-padron48576 ай бұрын

    god bless you brother

  • @kexinfang1093
    @kexinfang10933 жыл бұрын

    Thanks a lot!!!!!

  • @natachajouonang2710
    @natachajouonang27102 жыл бұрын

    Amazing and super helpful video! Extremely simple and easy to follow! But please, quick question: Why did you switch the Xi and Xbar at 7:51? This drastically changes the ending solution.

  • @malolanbalaji98

    @malolanbalaji98

    2 жыл бұрын

    When he removes the inner paranthesis, the term Xi becomes negative and Xbar becomes positive. So when you multiply it by (-ve)Beta, the signage of both terms reverses

  • @1UniverseGames
    @1UniverseGames3 жыл бұрын

    How can we find the intercept and slope value of B0 and B1

  • @reinier5355
    @reinier53553 жыл бұрын

    great video, my summary just gave the formula with the text: 'just remember this' hate that

  • @peehugupta5803
    @peehugupta58033 жыл бұрын

    thank you so so much :)))))

  • @egnos
    @egnos4 жыл бұрын

    2:24, where did you discuss why it makes sense to minimize the sum of squared residuals ?

  • @aakarshan01

    @aakarshan01

    4 жыл бұрын

    makes it more sensitive to bigger errors. And it's differentiable at all points. In the Mod function , it is not differentiable at the point it pivots up

  • @SuperYtc1

    @SuperYtc1

    3 жыл бұрын

    @@aakarshan01 but why not to power 1.5? why not to power 4? why is it exactly power 2?

  • @aakarshan01

    @aakarshan01

    3 жыл бұрын

    @@SuperYtc1 you can.but there is no need to. The differentiability is achieved in square. Why calculate a bigger number that could lead to problems since power 4 of a decimal number of more likely to break the minimum number limit of a float than a square. But in theory, you can

  • @mobileentertainment212
    @mobileentertainment212 Жыл бұрын

    In which video does he discuss why the we use squared residuals?

  • @girlstrends8204
    @girlstrends82045 ай бұрын

    thank you so much ...............

  • @jbstatistics

    @jbstatistics

    5 ай бұрын

    You are very welcome!

  • @alexa-lv6di
    @alexa-lv6di3 жыл бұрын

    How do we do this if we have three or more unknown parameters?

Келесі