20 Lagrange multipliers II

Lagrangian formulation

Remark

The Lagrangian combines the objective function \(f\) and the constraint function \(g\):

\[\mathcal{L}(x_1,\dots,x_n,\lambda) = f(x_1,\dots,x_n) - \lambda ( g(x_1,\dots,x_n) - c ).\]

This auxiliary function is useful because its critical points encode the \(n+1\) Lagrange equations:

\[\nabla \mathcal{L} = \langle f_{x_1} - \lambda g_{x_1}, \cdots, f_{x_n} - \lambda g_{x_n}, c - g \rangle.\]

The critical points (the stationary points) of the Lagrangian are the solutions to our optimization problem.

Method of Lagrange multipliers (Stewart, Clegg, and Watson 2020, 14.8.1)

We want to optimize \(f(x_1,\dots,x_n)\) subject to a constraint \(g(x_1,\dots,x_n) = c\).

Compute the gradient of the Lagrangian \(\mathcal{L}(x_1,\dots,x_n,\lambda) = f - \lambda (g-c)\),
Set \(\nabla \mathcal{L} = 0\) (a system of equations) and solve to find its stationary points, and
Plug the stationary points into \(f(x_1,\dots,x_n)\) to determine extreme values.

If you prefer to solve constrained optimization problems without using the Lagrangian formulation, that’s fine!

More examples

Example: Find the nearest point on the hyperbola \(x^2 - 5 y^2 = 1\) to the point \((2,0)\).

We take \(f(x,y) = (x - 2)^2 + y^2\) (since the square of a function, namely the distance function, will inherit its critical points), \(g(x,y) = x^2 - 5 y^2\), and \(c = 1\).

flowchart LR
  A("2(x-2) = 2λx <br> 2y = -10λy <br> x²-5y² = 1")
  A --> B("2(x-λx-2) = 0 <br> 2y (1+5λ) = 0 <br> x²-5y² = 1")
  B -->|"y=0"| C("2(x-λx-2) = 0 <br> x² = 1")
  B -->|"λ=-1/5"| D("2(6x/5-2) = 0 <br> x²-5y² = 1")
  C -->|"x=1"| E("-2(1+λ) = 0")
  C -->|"x=-1"| F("2(-3+λ) = 0")
  E --> G("(x,y,λ) = (1,0,-1)"):::solution
  F --> H("(x,y,λ) = (-1,0,3)"):::solution
  D --> I("x = 5/3 <br> 25/9-5y^2 = 1")
  I --> J("(x,y,λ) = (5/3, -4/(3√5), -1/5)"):::solution
  I --> K("(x,y,λ) = (5/3, 4/(3√5), -1/5)"):::solution

The bottom two give minima (\(f=\frac{7}{15}\)), and all are visualized below in red:

**What is the significance of the points that did not give the absolute minimum?**

Mathematica

This code solves the above system, and can be used to check your solutions in Homework.

f = (x - 2)^2 + y^2;
g = x^2 - 5 y^2;
c = 1;
L = f - λ (g - c);
gradL = Grad[L, {x, y, λ}];
solutions = Solve[gradL == 0, {x, y, λ}, Reals]
f /. solutions

Try modifying the code to solve problems in more variables!

Example: extrema of \(f(x,y) = x y e^{-x^2 - y^2}\), subject to \(g(x,y) = 2 x - y\) and \(c = 0\).

Since the constraint is quite simple, we solve \(y = 2x\) and substitute into the other Lagrange equations: \[ e^{-5x^2} \langle 2x(2x^2-1), x(4x^2-1) \rangle = \lambda \langle 2, -1 \rangle. \] Solving for \(\lambda\) in both equations and equating gives \(x(2x^2-1) e^{-5x^2} = x(1-8x^2) e^{-5x^2}.\) Hence we have \[ x(2x^2-1) = x(1-8x^2) \implies 2 x (5 x^2 - 1) = 0 \implies x = 0 \text{ or } x = \pm \tfrac{1}{\sqrt{5}}. \] Thus the stationary points of the Lagrangian are: \[(0, 0, 0), (\tfrac{1}{\sqrt{5}}, \tfrac{2}{\sqrt{5}}, \tfrac{3}{5 \sqrt{5} e}), (-\tfrac{1}{\sqrt{5}}, -\tfrac{2}{\sqrt{5}}, -\tfrac{3}{5 \sqrt{5} e}).\] The first is the minimum (\(f=0\)) and the last two are maxima (\(f=\frac{2}{5e}\)).

Homework

(Stewart, Clegg, and Watson 2020, sec. 14.8) #41, 43–45, 47, 54–55

You should also reflect on the bonus problem, §14.8.35.

Stewart, James, Daniel K. Clegg, and Saleem Watson. 2020. Calculus: Early Transcendentals. 9th ed. Cengage Learning.