Method of Lagrange Multipliers

Lagrange’s method is a powerful technique for finding the critical points of a function of two variables, $ f(x,y) $, when those variables are subject to a constraint.

Rather than searching for extrema over the entire plane, we restrict our search to the set of points that satisfy the constraint.

This approach allows us to locate constrained extrema without needing to parametrize the constraint explicitly.

How does it work?

Given a function $ f(x, y) $ subject to the constraint $ g(x, y) = 0 $, we construct a new function called the Lagrangian:

$$ L(x, y, \lambda) = f(x, y) + \lambda g(x, y) $$

Here, the parameter $ \lambda $ is known as the Lagrange multiplier.

To find the critical points of the Lagrangian, we take its partial derivatives and set them equal to zero. This yields the following system of equations:

$$
\begin{cases}
f_x(x,y) + \lambda g_x(x,y) = 0 \\
f_y(x,y) + \lambda g_y(x,y) = 0 \\
g(x, y) = 0
\end{cases}
$$

Here, $ f_x(x,y) = \frac{\partial f}{\partial x} $ and $ f_y(x,y) = \frac{\partial f}{\partial y} $ are the partial derivatives of $ f $ with respect to $ x $ and $ y $, respectively.

The multiplier $ \lambda $ remains an unknown and may take on either positive or negative values. This system is often written in an equivalent, more symmetric form:

$$
\begin{cases}
\frac{\partial f}{\partial x} = \lambda \frac{\partial g}{\partial x} \\
\frac{\partial f}{\partial y} = \lambda \frac{\partial g}{\partial y} \\
g(x, y) = 0
\end{cases}
$$

Solving this system yields the coordinates of the critical points $ (x, y) $ along the constraint curve where the function $ f(x, y) $ may attain a local maximum or minimum.

Once these points are identified, we must determine whether each corresponds to a maximum or minimum. There are two common ways to do this:

• Evaluate $ f(x, y) $ at each critical point and compare the resulting values.
• Analyze the second-order conditions using the Hessian matrix.

In essence, the Lagrangian method transforms a constrained optimization problem into a solvable system using tools from differential calculus.

Note: The method requires the constraint to be written in the standard form $ g(x,y)=0 $.

A Worked Example

Consider the function of two variables:

$$ f(x, y) = 3x + 4y $$

subject to the constraint that the point lies on a circle:

$$ x^2 + y^2 = 25 $$

Since Lagrange’s method requires the constraint in the form $ g(x,y)=0 $, we rewrite it as:

$$ x^2 + y^2 - 25 = 0 $$

We now seek the points on the circle $ x^2 + y^2 = 25 $ where $ f(x, y) = 3x + 4y $ is stationary - that is, where its rate of change in the constrained direction vanishes.

We begin by constructing the Lagrangian:

$$ L(x, y, \lambda) = 3x + 4y + \lambda(x^2 + y^2 - 25) $$

Next, we compute the partial derivatives and set them to zero:

$$ \frac{\partial L}{\partial x} = 3 + 2\lambda x = 0 $$

$$ \frac{\partial L}{\partial y} = 4 + 2\lambda y = 0 $$

This gives the system:

$$ \begin{cases}
3 + 2\lambda x = 0 \\
4 + 2\lambda y = 0 \\
x^2 + y^2 = 25
\end{cases}
$$

We now solve this system.

From the first equation, we isolate $ \lambda $:

$$ \lambda = -\frac{3}{2x} $$

Substitute into the second equation:

$$ \begin{cases}
\lambda = -\frac{3}{2x} \\
\lambda = -\frac{2}{y} \\
x^2 + y^2 = 25
\end{cases} $$

Equating the two expressions for $ \lambda $:

$$ -\frac{3}{2x} = -\frac{2}{y} \quad \Rightarrow \quad \frac{3}{2x} = \frac{2}{y} \Rightarrow 3y = 4x \Rightarrow y = \frac{4}{3}x $$

We substitute $ y = \frac{4}{3}x $ into the constraint:

$$ x^2 + \left(\frac{4}{3}x\right)^2 = 25 $$

$$ x^2 + \frac{16}{9}x^2 = 25 $$

$$ \frac{25}{9}x^2 = 25 $$

$$ x^2 = 9 \quad \Rightarrow \quad x = \pm 3 $$

If $x = 3$, then $y = 4$; if $x = -3$, then $y = -4$.

Thus, the stationary points are $ P_1 = (3, 4) $ and $ P_2 = (-3, -4) $.

To determine which is the maximum and which is the minimum, we evaluate the function at each point:

$$ f(3, 4) = 3 \cdot 3 + 4 \cdot 4 = 9 + 16 = 25 $$

$$ f(-3, -4) = -9 - 16 = -25 $$

So, $ P_1 = (3, 4) $ is the point of maximum value ( $ f(3,4) = 25 $ ), and $ P_2 = (-3, -4) $ is the point of minimum value ( $ f(-3,-4) = -25 $ ) under the constraint.

In this way, we’ve identified the points on the circle where $f(x, y)$ reaches its maximum and minimum - using only equations and derivatives, with no need to sketch the curve.

Evaluating the Augmented Hessian

To determine whether the critical points $ (3, 4) $ and $(-3, -4) $ correspond to a maximum or a minimum, we can use the augmented Hessian matrix, which incorporates both the objective function and the constraint.

$$
H_L =
\begin{pmatrix}
L_{xx} & L_{xy} & g_x \\
L_{yx} & L_{yy} & g_y \\
g_x & g_y & 0
\end{pmatrix}
$$

Where:

• $L_{xx} = \frac{\partial^2 L}{\partial x^2} = f_{xx} + \lambda g_{xx}$, and similarly for the other second-order terms.
• $g_x = \frac{\partial g}{\partial x}$

Note: The augmented Hessian differs from the classical Hessian, which is used to study the nature of critical points in unconstrained optimization. The classical Hessian is based solely on the second derivatives of the function $f(x, y)$: $$ H = \begin{pmatrix}
\frac{\partial^2 f}{\partial x^2} & \frac{\partial^2 f}{\partial x \partial y} \\
\frac{\partial^2 f}{\partial y \partial x} & \frac{\partial^2 f}{\partial y^2}
\end{pmatrix} $$ The augmented Hessian, by contrast, is designed for constrained optimization problems. It includes second derivatives of both the objective function and the constraint, as well as the first derivatives of the constraint, as encoded in the Lagrangian: $$
H_L =
\begin{pmatrix}
L_{xx} & L_{xy} & g_x \\
L_{yx} & L_{yy} & g_y \\
g_x & g_y & 0
\end{pmatrix}
$$

In this example, we compute the second-order partial derivatives of the Lagrangian $ L(x, y, \lambda) = 3x + 4y + \lambda(x^2 + y^2 - 25) $, and of the constraint $ g(x,y) = x^2 + y^2 - 25 $:

$$ L_{xx} = 2 \lambda \quad L_{xy} = L_{yx} = 0 \quad L_{yy} = 2 \lambda $$

$$ g_{xx} = g_{yy} = 2 \quad g_{xy} = 0 $$

$$ g_x = 2x \quad g_y = 2y $$

Substituting these into the augmented Hessian, we get:

$$ H_L = \begin{array}{ccc}
2\lambda & 0 & 2x \\
0 & 2\lambda & 2y \\
2x & 2y & 0
\end{array} $$

We now evaluate the matrix at each critical point by substituting the coordinates and computing the corresponding value of $ \lambda $.

We then calculate the determinant of the matrix. A positive determinant indicates a local constrained maximum, while a negative one indicates a local constrained minimum.

1] Critical point (3, 4)

We compute $ \lambda $ from the condition:

$$ 3 + 2\lambda \cdot 3 = 0 \quad \Rightarrow \quad \lambda = -\frac{1}{2} $$

Substituting into the matrix:

$$
H_L =
\left|
\begin{array}{ccc}
-1 & 0 & 6 \\
0 & -1 & 8 \\
6 & 8 & 0
\end{array}
\right|
$$

We compute the determinant using expansion along the first row:

$$ \Delta H_L =
-1 \cdot
\left|
\begin{array}{cc}
-1 & 8 \\
8 & 0
\end{array}
\right|
- 6 \cdot
\left|
\begin{array}{cc}
0 & -1 \\
6 & 8
\end{array}
\right|
$$

$$ \Delta H_L = -1( -1 \cdot 0 - 8 \cdot 8 ) - 6( 0 \cdot 8 - (-1) \cdot 6 ) $$

$$ \Delta H_L = -1(-64) - 6(6) = 64 - 36 = 28 $$

The determinant is positive, confirming that $ (3, 4) $ is a local constrained maximum.

2] Critical point (-3, -4)

We compute $ \lambda $ using:

$$ 3 + 2\lambda \cdot (-3) = 0 \quad \Rightarrow \quad \lambda = \frac{1}{2} $$

Substituting this value into the matrix gives:

$$
H_L =
\left|
\begin{array}{ccc}
1 & 0 & -6 \\
0 & 1 & -8 \\
-6 & -8 & 0
\end{array}
\right|
$$

We again expand along the first row:

$$ \Delta H_L = 1 \cdot
\left|
\begin{array}{cc}
1 & -8 \\
-8 & 0
\end{array}
\right|
- 6 \cdot
\left|
\begin{array}{cc}
0 & 1 \\
-6 & -8
\end{array}
\right|
$$

$$ \Delta H_L = 1( 0 - (-8)(-8) ) - 6( 0 - (-6) ) $$

$$ \Delta H_L = -64 - 36 = -100 $$

The determinant is negative, confirming that $ (-3, -4) $ is a local constrained minimum.

This analysis using the augmented Hessian matrix fully supports the classification we obtained earlier by directly evaluating the function values.

Example 2

We want to determine the minimum and maximum values of the function

$$ f(x,y) = x-2y $$

subject to the constraint of a circle:

$$ g(x,y) = \{(x,y) \in R^2 : x^2+y^2 \le 3 $$

We begin by rewriting the constraint in the form $ g(x,y)=0 $:

$$ x^2+y^2 - 3 = 0 $$

Our strategy is to analyze the boundary of the feasible region.

To find candidate points where the minimum and maximum may occur, we first attempt to solve the following system:

$$ \begin{cases} g_x(x,y) = 0 \\ g_y(x,y) = 0 \\ g(x,y)=0 \end{cases} $$

$$ \begin{cases} 2x = 0 \\ 2y = 0 \\ x^2+y^2 - 3=0 \end{cases} $$

$$ \begin{cases} x = 0 \\ y = 0 \\ x^2+y^2=3 \end{cases} $$

This system has no solution.

Therefore, we proceed by solving the system given by the method of Lagrange multipliers:

$$ \begin{cases} f_x(x,y) = \lambda \cdot g_x(x,y) \\ f_y(x,y) = \lambda \cdot g_y(x,y) \\ g(x,y)=0 \end{cases} $$

$$ \begin{cases} 1 = 2 \lambda x \\ -2 = 2 \lambda y \\ x^2+y^2 - 3=0 \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ x^2+y^2 - 3=0 \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ ( \frac{1}{2 \lambda} )^2+( - \frac{1}{ \lambda } )^2 =3 \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ \frac{1}{4 \lambda^2} + \frac{1}{ \lambda^2 } =3 \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ \frac{5}{4 \lambda^2} =3 \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ \lambda^2 = \frac{5}{12} \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ \lambda = \pm \frac{\sqrt{5}}{ \sqrt{12}} \end{cases} $$

$$ \begin{cases} x = \frac{1}{2 \lambda} \\ y = - \frac{1}{ \lambda } \\ \lambda = \pm \frac{\sqrt{5}}{ 2\sqrt{3}} \end{cases} $$

Now that we have found $ \lambda = \pm \frac{\sqrt{5}}{ 2\sqrt{3}} $, we substitute these values back into the first two equations to compute the coordinates of the candidate points $(x,y)$.

For $ \lambda = \frac{\sqrt{5}}{ 2\sqrt{3}} $:

$$ x = \frac{1}{2 \lambda} $$

$$ x = \frac{1}{2 \frac{\sqrt{5}}{ 2\sqrt{3}}} $$

$$ x = \frac{1}{\frac{\sqrt{5}}{ \sqrt{3}}} $$

$$ x = \frac{ \sqrt{3} }{ \sqrt{5}} $$

$$ y = - \frac{1}{ \lambda } $$

$$ y = - \frac{1}{ \frac{\sqrt{5}}{ 2\sqrt{3}} } $$

$$ y = - \frac{ 2\sqrt{3} }{ \sqrt{5}} $$

Thus, the first candidate point is $ (x;y)= ( \frac{ \sqrt{3} }{ \sqrt{5}} ; - \frac{ 2\sqrt{3} }{ \sqrt{5}} ) $.

For $ \lambda = - \frac{\sqrt{5}}{ 2\sqrt{3}} $:

$$ x = \frac{1}{2 \lambda} $$

$$ x = \frac{1}{2 (- \frac{\sqrt{5}}{ 2\sqrt{3}} ) } $$

$$ x = - \frac{1}{\frac{\sqrt{5}}{ \sqrt{3}}} $$

$$ x = - \frac{ \sqrt{3} }{ \sqrt{5}} $$

$$ y = - \frac{1}{ \lambda } $$

$$ y = - \frac{1}{ - \frac{\sqrt{5}}{ 2\sqrt{3}} } $$

$$ y = \frac{ 2\sqrt{3} }{ \sqrt{5}} $$

Thus, the second candidate point is $ (x;y)= ( - \frac{ \sqrt{3} }{ \sqrt{5}} ; \frac{ 2\sqrt{3} }{ \sqrt{5}} ) $.

We now substitute these coordinates into the function $f(x,y)$ to determine which corresponds to the maximum value and which to the minimum:

$$ f( \frac{ \sqrt{3} }{ \sqrt{5}} ; - \frac{ 2\sqrt{3} }{ \sqrt{5}}) = \frac{ \sqrt{3} }{ \sqrt{5}} + \frac{ 4\sqrt{3} }{ \sqrt{5}} = \frac{5 \sqrt{3}}{ \sqrt{5} } = \sqrt{15} $$

$$ f( - \frac{ \sqrt{3} }{ \sqrt{5}} ; \frac{ 2\sqrt{3} }{ \sqrt{5}}) = - \frac{ \sqrt{3} }{ \sqrt{5}} - \frac{ 4\sqrt{3} }{ \sqrt{5}} = - \frac{5 \sqrt{3}}{ \sqrt{5} } = - \sqrt{15} $$

It is therefore evident that the maximum value of $f(x,y)$ subject to the constraint $g(x,y)$ occurs at $ (x;y)= ( \frac{ \sqrt{3} }{ \sqrt{5}} ; - \frac{ 2\sqrt{3} }{ \sqrt{5}} ) $, while the minimum value occurs at $ (x;y)= ( - \frac{ \sqrt{3} }{ \sqrt{5}} ; \frac{ 2\sqrt{3} }{ \sqrt{5}} ) $.

And so on.