This was an amazing explanation of the geometric solution to OLS and how the normal equations arise.

Thank you for the clear and helpful explanation. If I could click on the "like" button more than once, I would.

So happy to find your video! Love the handwriting and clear explanation!

that is brilliant. linear algebra has always been kind of hard for me, but i understood this pretty quickly.

Very concise and helpful. Thank you.

Very clear explanation. Thanks!

Very well done, thank you sir!

Thanks a lot. I was really enlightened by your very clear explanations.

Thank you, you give a wonderful explanation!

Super nice video! I enjoy your skill in explaining :)

Thank you, this was very helpful.

Thank You Sir. This video cleared my all doubts.

Thanks, that was very helpful!

Understood on the second watch. Thank you very much!

Very well explained !Thank you Sir,

Really helpful! thank you so much

Very helpful, thank you !

Great explanation.

Thank you! It was so helpful!

Excellent. Thank you.

great work to use geometry to explain the massive equation!!!

Good! Thank you!

thank you

THANKS

Hello sir ,thats a very helpful video ..but I just did not understand that step at the 8:05 minutes ,when you find the result to X1 and X2 ..could you please explain to me ..

@mak5386

6 жыл бұрын

Late ,but still.. A'Ax=A'b is what u need to solve; u have got both A'A and A'b ,u could easily solve the 2 equations by hand ,or by taking the inverse of [4 10,10 30].

@H4nek

4 жыл бұрын

To solve x1, x2, you can create 2 scalar equations out of that matrix one (ATAx = ATb). You'll have: 4x1 + 10x2 = 12 10x1 + 30x2 = 36 And simply solve it.

Excellent explanation! But how is this related to Moore-Penrose pseudoinverse? I know MP pseudoinverse is supposed to help us finding a least square solution, just like you're trying to, but MP requires some more matrix computation. Are those two methods equivalent?

@lorenzosadun565

9 жыл бұрын

***** If the columns are linearly independent, then A^T A is invertible, and the solution to A^T A x = A^T b is x = (A^T A)^{-1} A^T b, and we call the matrix (A^T A)^{-1} A^T the pseudoinverse of A. However, when the columns of A are linearly dependent, then there are many least-squares solutions, since the matrix A^T A is singular.

Thank you.

Thanks!

Thank you Sir

Is the weight matrix G, you mention at 9:09 onwards, the Mahalanobis matrix? (Or Mahalanobis distance called as well) Overall, nice tutorial!

@lorenzosadun565

8 жыл бұрын

+mmuuuuhh I'm not too familiar with the Mahalanobis matrix (beyond what I just looked up on Wikipedia), but I don't think they're the same. The Mahanobis matrix refers to the covariance of two measurements. We're assuming that all measurements are independent (insofar as G is diagonal). However, it may be that, in the case of independent measurements with different variances, the Mahanobis matrix is the inverse of G.

@ 8:11 where do the numbers at the bottom come from? they just appears out of nowhere

I follow how solving Ax = b_parallel is equivalent to solving A^T x = A^T b (because A^T b_perp = 0), but I don't understand how solving Ax = b (no exact solutions for x) isn't also equivalent to solving A^T x = A^T b? Because all you have done is multiplied each side by the transpose of A. I understand that there can't be an exact solution for x, if b is not in the column space of A, because there are more independent equations than unknowns, but where is the error in a solution like Ax = b, therefore A^T A x = A^T b, therefore x = (A^T A)^{-1} A^T b (assuming A^T A is invertible)?

@lorenzosadun565

8 жыл бұрын

+chickensandwiche The error is that you're assuming that Ax=b. IF there is a true solution with Ax=b, then that is ALSO a solution to A^T A x = A^T b, and so is a least-squares solution. (Having error=0 makes the size of the error as small as possible). But there are plenty of situations where there are solutions to A^T A x = A^T b but not to A x = b. Just take (say) A = [1 1; 1 2; 2 2] and b=[2; 3; 5]. (That's MATLAB notation, where ";" means "carriage return")

could someone tell me,, Why a transpose * b equals the inner product of a and b ?

@lorenzosadun565

9 жыл бұрын

Jojo Kha The inner product of a and b is \sum a_i b_i = a^T b.

@jafaralkhatib9947

9 жыл бұрын

Lorenzo Sadun Many thanks

I think A^T should be in the rowspace instead of column space.

Eng kaahiye