Lecture 9: Four Ways to Solve Least Squares Problems

Description

In this lecture, Professor Strang details the four ways to solve least-squares problems. Solving least-squares problems comes in to play in the many applications that rely on data fitting.

Summary

1. Solve \(A^{\mathtt{T}} Ax = A^{\mathtt{T}}b\) to minimize \(\Vert Ax - b \Vert^2\)
2. Gram-Schmidt \(A = QR\) leads to \(x = R^{-1} Q^{\mathtt{T}}b\).
3. The pseudoinverse directly multiplies \(b\) to give \(x\).
4. The best \(x\) is the limit of \((A^{\mathtt{T}}A + \delta I)^{-1} A^{\mathtt{T}}b\) as \(\delta \rightarrow 0\).

Related section in textbook: II.2

Instructor: Prof. Gilbert Strang