Minimizers

Here, we consider a general optimization problem where and is any function.

A point is a global minimizer of if for all .

A point is a local minimizer of if there exists a such that for all where .

Suppose is twice continuously differentiable. The Hessian of at a point is

Suppose is differentiable. If is an interior point of and local minimizer of , then If furthermore is twice continuously differentiable, then the Hessian is positive semidefinite.

Proof:
Let with . Define by for some such that is a global minimum in its neighbourhood of radius . Then is a global minimum of . We get since by minimality, . But also, so for the derivative to exist we must have .

Now suppose . Since is twice continuously differentiable, is continuous and there thus exists a such that on . By Taylor expansion, for every there exists a with and since we get , contradicting the minimality of . Thus .

Since all directional derivatives are zero we have . We also know that where is the Hessian, meaning so is indeed positive semidefinite.