Understanding Saddle Points: A Deep Dive into Their Nature, Mixed Second Derivative Test, and Critical Points

In multivariable calculus, understanding the nature of critical points—whether they are local minima, local maxima, or saddle points—is crucial. Saddle points, in particular, occupy a special place due to their unique behavior. They are neither minima nor maxima but exhibit characteristics of both, curving upwards in one direction and downwards in another. This blog post will cover the concept of saddle points, using examples and graphical representations, as well as delve into the mixed second derivative test (also known as the Hessian determinant test) and its role in determining saddle points.

We will also, explore the mathematical foundations, provide code examples for visualization, and explain when and why a function has a saddle point based on the second derivative test.

1. What Is a Saddle Point?

A saddle point is a type of critical point in a multivariable function where the function behaves like a minimum in one direction and like a maximum in another. If you visualize a saddle point, it resembles the shape of a horse's saddle—hence the name. Saddle points occur in functions of two or more variables.

Definition of a Critical Point

A critical point of a function

$$( f(x, y) )$$

is a point where the first partial derivatives of the function are zero:

$$f_x(x_0, y_0) = 0 \quad \text{and} \quad f_y(x_0, y_0) = 0$$

or,

$$\frac{\partial f}{\partial x}(x_0, y_0) = 0 \quad \text{and} \quad \frac{\partial f}{\partial y}(x_0, y_0) = 0$$

At these points, the slope of the surface is zero in both the (x)- and (y)-directions. Critical points can be classified as local minima, local maxima, or saddle points, depending on the behavior of the function in the neighborhood around that point.

What Makes a Saddle Point Special?

At a saddle point, the function neither exhibits a strict minimum nor a maximum, yet its slope is zero at the critical point. The defining characteristic of a saddle point is that the function curves upwards (positive curvature) in one direction and downwards (negative curvature) in another. Mathematically, this results in an indefinite form of the Hessian determinant, which we'll cover shortly.

Visual Intuition

Imagine walking along a mountain ridge that gently slopes upward in one direction but slopes downward in the other. The point where you transition between ascending and descending is analogous to a saddle point. In mathematical terms, the function increases in some directions and decreases in others near the saddle point.

Let’s illustrate this with an example.

2. Example of a Saddle Point Function

To better understand saddle points, let's consider an example of a function that exhibits a saddle point. Consider the following function of two variables:

$$f(x, y) = 3x^2 - 2y^2$$

This function has a critical point at (0, 0), and it features a saddle point at that location. To analyze this:

Step 1: Find the Critical Points

To find the critical points, we need to compute the first partial derivatives and set them equal to zero.

1. First partial derivative with respect to x:

$$f_x = \frac{\partial}{\partial x}(3x^2 - 2y^2) = 6x$$

2. First partial derivative with respect to y:

$$f_y = \frac{\partial}{\partial y}(3x^2 - 2y^2) = -4y$$

Setting both partial derivatives equal to zero:

$$6x = 0 \quad \text{and} \quad -4y = 0$$

Thus, the only critical point is

$$(x_0, y_0) = (0, 0)$$

Step 2: Classify the Critical Point Using the Mixed Second Derivative Test

To classify the critical point, we need to compute the second-order partial derivatives:

1. Second partial derivative with respect to x:

$$f_{xx} = \frac{\partial^2 f}{\partial x^2} = 6$$

2. Second partial derivative with respect to y:

$$f_{yy} = \frac{\partial^2 f}{\partial y^2} = -4$$

3. Mixed partial derivative:

$$f_{xy} = \frac{\partial^2 f}{\partial x \partial y} = 0$$

The Hessian determinant D is then calculated as follows:

$$D = f_{xx} \cdot f_{yy} - (f_{xy})^2 = (6)(-4) - (0)^2 = -24$$

Since D < 0, the point (0,0) is a saddle point.

3. The Mixed Second Derivative Test and Hessian Determinant

The mixed second derivative test, or Hessian determinant test, is used to classify critical points as local minima, local maxima, or saddle points. The test involves calculating the second-order partial derivatives of the function and using them to compute the Hessian determinant.

The Hessian Matrix

The Hessian matrix is a square matrix of second-order partial derivatives of a function. For a function f(x,y), the Hessian matrix H is defined as:

$$H = \begin{bmatrix} f_{xx} & f_{xy} \\ f_{yx} & f_{yy} \end{bmatrix}$$

Here:

$$f_{xx} = \frac{\partial^2 f}{\partial x^2}​$$

$$f_{yy} = \frac{\partial^2 f}{\partial y^2}$$

$$f_{xy} = f_{yx} = \frac{\partial^2 f}{\partial x \partial y}$$

• since mixed partial derivatives are equal under most conditions, as per Clairaut’s theorem.

The determinant of the Hessian matrix is given by:

$$D = f_{xx} f_{yy} - (f_{xy})^2$$

Classification of Critical Points Using the Hessian Determinant

1. If D>0:

$$f_{xx} > 0,$$

• the critical point is a local minimum.

$$f_{xx} < 0,$$

• the critical point is a local maximum.

1. If D<0:

   • The critical point is a saddle point.

2. If D=0:

   • The test is inconclusive, and higher-order derivatives must be considered.

Let’s take another example to apply this test.

Example: Finding the Saddle Point

Consider the function:

$$f(x, y) = x^3 - 3xy^2$$

Step 1: Find the Critical Points

1. First partial derivative with respect to x:

$$f_x = 3x^2 - 3y^2$$

2. First partial derivative with respect to y:

$$f_y = -6xy$$

Set these equal to zero:

$$3x^2 - 3y^2 = 0 \quad \text{and} \quad -6xy = 0$$

From the first equation,

$$x^2 = y^2,$$

which gives

$$x=y \quad or\quad x=−y.$$

From the second equation, either

$$x=0 \quad or \quad y=0.$$

• If x=0, then y=0.

• If y=0, then x=0.

Thus, the critical point is (0,0).

Step 2: Classify the Critical Point Using the Mixed Second Derivative Test

We compute the second-order partial derivatives:

1. Second partial derivative with respect to x:

$$f_{xx} = 6x$$

2. Second partial derivative with respect to y:

$$f_{yy} = -6x$$

3. Mixed partial derivative:

$$f_{xy} = -6y$$

At (0,0), we have:

$$f_{xx} = 0, \quad f_{yy} = 0, \quad f_{xy} = 0$$

The Hessian determinant is:

$$D = (0)(0) - (0)^2 = 0$$

Since D=0, the test is inconclusive.

Step 3: Further Analysis

Since the Hessian test is inconclusive, we need to analyze the behavior of the function in different directions. By testing along different paths, we find that the point (0,0) is a saddle point.

This demonstrates that when the Hessian determinant is zero, further analysis is required to determine the nature of the critical point.

4. Critical Points at (0,0): Are They Always Saddle Points?

No, critical points at (0,0) are not always saddle points. Here are the reasons:

1. Definition of Critical Points: A critical point is a point where the gradient (first derivatives) of a function is zero or undefined. It could be a local minimum, local maximum, or saddle point.

Nature of Critical Points: The classification of a critical point depends on the second derivative test (Hessian matrix) or the nature of the function around that point:

• Local Minimum: If the Hessian is positive definite (all eigenvalues positive) at (0,0), the point is a local minimum.

• Local Maximum: If the Hessian is negative definite (all eigenvalues negative) at (0,0), the point is a local maximum.

• Saddle Point: If the Hessian has both positive and negative eigenvalues, then (0,0) is a saddle point.

Whether a critical point at (0,0) is a saddle point depends on the function’s behavior in the neighborhood of that point, and not merely on its location.

Example 1: Local Minimum at (0,0)

Consider the function:

$$f(x, y) = x^2 + y^2$$

The critical point is at (0,0), and we compute the second-order partial derivatives:

$$f_{xx} = 2, \quad f_{yy} = 2, \quad f_{xy} = 0$$

The Hessian determinant is:

$$D = (2)(2) - (0)^2 = 4 > 0$$

Since

$$D > 0 \quad and\quad f_{xx} > 0$$

(0,0) is a local minimum, not a saddle point.

Example 2: Local Maximum at (0,0)

Consider the function:

$$f(x, y) = -x^2 - y^2$$

The critical point is again at (0,0), and we compute the second-order partial derivatives:

$$f_{xx} = -2, \quad f_{yy} = -2, \quad f_{xy} = 0$$

The Hessian determinant is:

$$D = (-2)(-2) - (0)^2 = 4 > 0$$

Since

$$D > 0 \quad and \quad f_{xx} < 0$$

(0,0) is a local maximum, not a saddle point.

Conclusion on (0,0)

The critical point at (0,0) can be a local minimum, local maximum, or a saddle point, depending on the function. The location (0,0) alone does not determine whether it is a saddle point. We must analyze the function's behavior using tools like the Hessian determinant or graphical analysis to correctly classify the critical point.

5. Alternative Methods to Identify Saddle Points Without Using the Hessian Determinant/Second-Order Derivatives

Although the second derivative test is commonly used, it is not always necessary to rely on it to identify saddle points. Several methods can be used to detect saddle points without computing the second-order derivatives:

1. Graphical Analysis

Plotting the function in 3D often gives an intuitive understanding of whether a point is a saddle point, minimum, or maximum. Saddle points typically appear as regions where the surface curves upward in one direction and downward in another.

2. Path Analysis

One can analyze the function along different paths through the critical point. For example, examining the function along the ( x )-axis and the ( y )-axis may reveal if the point behaves like a maximum in one direction and a minimum in another, which is characteristic of a saddle point.

3. Contour Plots

Contour plots can help visualize how the function behaves near the critical points. If the contours change shape or orientation around a critical point, it suggests that the point might be a saddle point. Saddle points typically appear where the level curves change direction from elliptic to hyperbolic.

6. Graphing Saddle Points Using Code

Let’s visualize the saddle point of the function

$$f(x, y) = x^3 - 3xy^2$$

using Python and Matplotlib. This will help us see how the function behaves around the saddle point.

Python Code to Visualize the Saddle Point:

import numpy as np
import matplotlib.pyplot as plt
from mpl_toolkits.mplot3d import Axes3D

# Define the function
def f(x, y):
    return 3*x**2 - 2*y**2

# Generate x and y values
x = np.linspace(-2, 2, 400)
y = np.linspace(-2, 2, 400)
x, y = np.meshgrid(x, y)

# Calculate z values
z = f(x, y)

# Create 3D plot
fig = plt.figure(figsize=(10, 8))
ax = fig.add_subplot(111, projection='3d')

# Plot surface
ax.plot_surface(x, y, z, cmap='coolwarm', edgecolor='none')

# Saddle point at (0,0)
ax.scatter(0, 0, 0, color='r', s=100)

ax.set_xlabel('X')
ax.set_ylabel('Y')
ax.set_zlabel('Z')
ax.set_title('Saddle Point at (0, 0) for f(x, y) = 3x^2 - 2y^2')

plt.show()

Output image

Here is the 3D plot of the function $$f(x, y) = 3x^2 - 2y^2$$ , showing the saddle point at (0, 0) . The surface curves upward along the ( x )-axis and downward along the ( y )-axis, illustrating the characteristic saddle shape. The red dot highlights the saddle point at (0, 0). To find a detailed explanation on the code blocks of the above graph please refer here - graphing-saddle-point.

7. Conclusion

In this blog post, we have explored saddle points in depth, covering the following key points:

• Saddle Points are critical points where the surface curves upwards in one direction and downwards in another.

• Mixed Second Derivative Test: The Hessian determinant test helps classify critical points as local minima, local maxima, or saddle points.

• Critical Points at (0,0) are not always saddle points; their classification depends on the behavior of the function in the neighborhood of that point.

• Alternative Methods: Graphical analysis, contour plots, and path analysis can help identify saddle points without relying on the Hessian determinant.

By understanding these concepts and using the methods discussed, we can classify critical points and identify saddle points in functions of multiple variables.