Least squares

Definition: Least squares problem

An optimization problem is called a least squares problem whenever it is of the form for some and .

Remark

All least squares problems are quadratic since

Theorem

A point is optimal for if and only if Proof:
We have where , and . is thus positive semidefinite, so by a previous theorem is optimal if and only if