Lagrange Multipliers

Summary

The geometry of Lagrange multipliers is explored in the context of the
optimization problem for y e^x on an ellipse. Solutions are also obtained
numerically using fsolve.

> with(plots):

Warning, the name changecoords has been redefined

The Function f Being Optimized and the Constraint

The function being optimized.

> f := y*exp(x);

The constraint, an ellipse.

> constr1 := x^2+x*y+4*y^2 -1=0;

The Constraint Set With Typical Level Curves of f

> gr1 := implicitplot(constr1,x=-2..2,y=-2..2,color=blue):

> display(gr1);

The geometric interpretation of Lagrange Multipliers concerns the interplay
between level curves of f and the constraint set. So we begin by looking at level curves.
The level curve f = -2.

> gr3 := implicitplot(f=-2,x=-2..2,y=-2..2,color=plum):

> display({gr1,gr3});

> gr4 := implicitplot(f=-.5,x=-2..2,y=-2..2,color=sienna):

The level with value -.5 meets the ellipse obliquely.
So, for example at the point of intersection near the y axis, moving to the left along the ellipse increases the value of the function while moving right decreases the value.

> display({gr1,gr3,gr4});

Setting Up and Solving the Lagrange Multiplier Condition

The usual auxiliary function in the method of Lagrange Multipliers.

> h := f - lambda * lhs(constr1);

> eqn1 := diff(h,x) = 0;

> eqn2 := diff(h,y) = 0;

The optional third argument in the fsolve call allows us to only seek solutions in
a certain range. Our choice of range here was suggested by the pictures above.

> fsolve({eqn1,eqn2,constr1},{x,y,lambda},{x=-1.5..1.5,y=-.5 .. .5});

These statements illustrate how to conveniently extract various numbers from the
solution set.

> solnset := %;

>

The exact order in which x,y, and lambda may vary from machine to machine or time to time.

> x_val := rhs(solnset[1]);

> y_val := rhs(solnset[3]);

Substituting a set of assignments bypasses the uncertainty of the order in which
x,y, and lambda appear.

> f_val :=evalf(subs(solnset,f));

The Basic Geometry of Lagrange Multipliers

> gr6 := implicitplot(f=f_val,x=-2..2,y=-2..2,color=red):

> display({gr1,gr3,gr4,gr6});

At a local extremum, the level curve of f must be tangent to the constraint set.
For if not, there would be a direction of motion along the constraint set for which the
dot product with grad f would be positive. So f would increase in that direction and
decrease in the opposite direction.

This is the origin of the Lagrange multiplier condition that
grad f = lambda grad g
at x0 if f restricted to the constraint set g = constant has a local extremum at x0 .

(Technically, we also need that grad(g) be nonzero at x0 , so that there is a smooth
curve near x0 describing a component of the constraint set near x0 .)

Finding the Maximum

Now we look for the maximum of f on the ellipse.

> solnset2 := fsolve({eqn1,eqn2,constr1},{x,y,lambda},{x=0..1.5,y=0 .. .5});

> x_val := rhs(solnset2[1]);

> y_val := rhs(solnset2[3]);

> f_val2 := evalf(subs(solnset2,f));

> gr7 := implicitplot(f=f_val2,x=-2..2,y=-2..2,color=green):

> display({gr1,gr7});

>

>