Nonlinear Programming Supplementary Notes

Graphics of parent functions

This is just to recall some basic graphs, used to quick test some points.

y=ax+b

y=ax^2+bx+c

$y=\frac{k}{x}$$y=\frac{k}{x}$

$y=a^x$$y=a^x$

$y=lo{g}_{a}x$$y=log_a x$

$y=x^a$$y=x^a$

Introduction to NLP

Example 1. Profit Maximization

Example 2. Product Maximization

Example 3.

Example 4.

Convex and Concave Functions

Example 5. Check cvx or ccv of $x^2$$x^2$

Example 6. check cvx $e^x$$e^x$

E7. check cvx $x^{\frac{1}{2}}$$x^\frac{1}{2}$ , $x\ge 0$$x\ge0$

E8. $f(x)=x^2+e^x$$f(x)=x^2+e^x$

Easy way: sum of convex is convex, so f(x) is convex.

Therorem 1

For maximization problem, if function is concave, local max = global max = optimal solution

For minimization problem, if function is convex, local min = global min = optimal solution

E9. check convexity of $f(x)=ax+b$$f(x)=ax +b$

E10. Hessian of of $f(x_1,x_2)=x_1^3+2x_1{x}_2+x_2^2$$f(x_1,x_2) = x_1^3 + 2x_1x_2 + x_2^2$

Definition 1 [Hessian, PM, LPM]

Theorem 2 [convex by PM, concave by PM]

E11. CVX or CCV $f(x_1,x_2)=x_1^2+2x_1{x}_2+x_2^2$$f(x_1,x_2) = x_1^2 + 2x_1x_2 + x_2^2$

E12. CVX or CCV $f(x_1,x_2)=-x_1^2-2x_1{x}_2-2x_2^2$$f(x_1,x_2) = -x_1^2 - 2x_1x_2 -2 x_2^2$

E13. CVX or CCV $f(x_1,x_2)=x_1^2-3x_1{x}_2+2x_2^2$$f(x_1,x_2) = x_1^2 - 3x_1x_2 + 2x_2^2$

E14. CVX or CCV $f(x_1,x_2,x_3)=x_1^2+x_2^2+2x_3^2-x_1{x}_2-x_2{x}_3-x_1{x}_3$$f(x_1,x_2,x_3) = x_1^2 +x_2^2 + 2x_3^2 - x_1x_2 -x_2x_3 - x_1x_3$

E15. CVX/CCV $f(x)=x^3,S=[0,\mathrm{\infty }]$$f(x) = x^3, S=[0,\infin]$

E16.$f(x)=x^3,S\in R$$f(x) = x^3, S\in R$

E17.$f(x)=\frac{1}{x},S\in (0,\mathrm{\infty })$$f(x) = \frac{1}{x}, S\in (0,\infin)$

E18. $f(x)=x^a(0 \le a\le 1);S\in (0,\infty )$$f (x)= x^a (0  \le a \le  1); S \in  (0, ∞)$

E19. $f(x) =lnx;S \in (0,\infty )$$f (x)  = ln x \ ; S  \in (0, ∞)$

E20. $f(x1,x2) =x_1^3 +3x_1{x}_2 +x_2^2;S\in R2$$f (x1, x2)  = x^3_1  + 3x_1x_2  + x_2^2; S \in  R2$

E21. $f(x_1,x_2)= x_1^2+ x_2^2;S =R2$$f (x_1, x_2) =  x^2_1 + x^2_2; S  = R2$

E22. CVX / CCV

E23. CVX / CCV

Solving NLPs with One Variable

Theorem 3

What happens if $f^{\prime }(x)=0,f^{″}(x)=0$$f'(x)=0, f''(x) =0$ ?

Both side neibours must lower or higher

E.24 Profit Maximization by Monopolist

E25. Product Pricing

E26. Monopolist Pricing

E27. find the opt to $max{x}^3,\text{where}x\in [-1,1]$$max \ x^3, \text{where} \ x\in [-1,1]$

E28. Find the optimal solution

Unconstrained NLPs with multiple variables

Theorem 4

This means, if we want to find a local maximum or local minimum, we must first find all points where the $f^{\prime }=0$$f' = 0$

Definition 2

All the points we find by doing $f^{\prime }=0$$f'=0$ are stationary points.

In the single variable case, if we do not has any constraints, we also did this:

1. first we calculated $f^{\prime }$$f'$ then we find several points, let’s say x_1, x_2.
2. then we calculated $f^{″}$$f''$ if $f^{″}(\overline{x})>0$$f''(\bar x) >0$ $\overline{x}$$\bar x$ is local minimum. (if $f^{″}(\overline{x})<0$$f''(\bar x) <0$, local max)
3. if $f^{\prime }(\overline{x})=0\mathrm{\&}{f}^{″}(\overline{x})=0$$f'(\bar x) = 0 \& f''(\bar x) = 0$ then it is a saddle point.

Theorem 5