Eigenvalues and Eigenvectors

Definition

• Eigenvector: An eigenvector of a square matrix \(A\) is a nonzero vector \(v\) such that when \(A\) is multiplied by \(v\), the direction of \(v\) remains unchanged. Mathematically, this relationship is defined as \(Av = \lambda v\), where \(\lambda\) is a scalar known as the eigenvalue associated with the eigenvector \(v\).

• Eigenvalue: The scalar \(\lambda\) in the equation \(Av = \lambda v\) is the eigenvalue corresponding to the eigenvector \(v\). It represents the factor by which the eigenvector is scaled during the transformation.

How to Calculate Eigenvalues and Eigenvectors

The calculation involves several steps:

1. Characteristic Equation: Start by finding the characteristic equation of matrix \(A\), which is obtained by subtracting \(\lambda\) times the identity matrix \(I\) from \(A\) and setting the determinant to zero: \(\det(A - \lambda I) = 0\).

2. Solve for \(\lambda\): Solve the characteristic equation for \(\lambda\). The solutions are the eigenvalues of \(A\).

3. Find Eigenvectors: For each eigenvalue \(\lambda\), solve the equation \((A - \lambda I)v = 0\) for vector \(v\). Each non-trivial solution \(v\) is an eigenvector associated with \(\lambda\).

Example

Let’s illustrate these concepts with a simple example. Consider a \(2 \times 2\) matrix \(A\):

\[ A = \begin{pmatrix} 4 & 1 \\ 2 & 3 \end{pmatrix} \]

Calculations in R

To calculate eigenvalues and eigenvectors in R, we can use the eigen() function. This function takes a square matrix as input and returns a list containing the eigenvalues and eigenvectors of that matrix. Let’s apply this to our example matrix \(A\).

# Define the matrix A
A <- matrix(c(4, 1, 2, 3), nrow = 2, byrow = TRUE)

# Calculate eigenvalues and eigenvectors
eigen_result <- eigen(A)

# Display the eigenvalues
print(eigen_result$values)

## [1] 5 2

# Display the eigenvectors
print(eigen_result$vectors)

##           [,1]       [,2]
## [1,] 0.7071068 -0.4472136
## [2,] 0.7071068  0.8944272

Visualization

Eigenvalues and eigenvectors play a crucial role in understanding the behavior of linear transformations represented by matrices. This section provides a visual intuition behind these concepts using R.

First, let’s load the libraries and define a function to plot vectors, which will be used to display eigenvectors and their transformations.

library(ggplot2)
library(gridExtra) # For arranging the plots

# Define a function to plot vectors (eigenvectors or transformed vectors)
plot_vectors <- function(vectors, title) {
  df <- data.frame(x = c(0, 0), y = c(0, 0), 
                   dx = c(vectors[1, 1], vectors[1, 2]), 
                   dy = c(vectors[2, 1], vectors[2, 2]))
  
  p <- ggplot() + 
    geom_segment(aes(x = x, y = y, xend = x + dx, yend = y + dy), 
                 data = df, arrow = arrow(type = "closed"), color = "red") +
    xlim(-1, 5) + ylim(-1, 5) +
    labs(title = title) + 
    theme_minimal()
  
  return(p)
}

Now, we visualize the eigenvectors of \(A\), the effect of \(A\) on a set of grid points, and how \(A\) transforms its eigenvectors.

# Plot A's eigenvectors
p1 <- plot_vectors(eigen_result$vectors, "Eigenvectors of A")

# We simulate the effect of A by transforming a set of grid points
# Define a grid of points
x <- seq(-1, 1, length.out = 10)
y <- seq(-1, 1, length.out = 10)
grid <- expand.grid(x = x, y = y)

# Transform each point by matrix A
transformed_grid_A <- as.data.frame(t(A %*% rbind(grid$x, grid$y)))

# Plotting the effect of A on points
p2 <- ggplot() +
  geom_point(aes(x, y), data = grid, color = "blue", alpha = 0.5) +
  geom_point(aes(V1, V2), data = transformed_grid_A, color = "red", alpha = 0.5) +
  xlim(-5, 5) + ylim(-5, 5) +
  labs(title = "Effect of A on Points") +
  theme_minimal()

# Plot transformed eigenvectors
transformed_vectors <- A %*% eigen_result$vectors
p3 <- plot_vectors(transformed_vectors, "Transforming A's Eigenvectors by A")

# Arrange the plots side by side
grid.arrange(p1, p2, p3, ncol = 3)

To further our understanding, we examine how a different matrix \(B\) affects the same set of points and eigenvectors of \(A\).

# Define the matrix B
B <- matrix(c(3, 3, 4, 1), nrow = 2, byrow = TRUE)

# Plot eigenvectors of A
p1 <- plot_vectors(eigen_result$vectors, "Eigenvectors of A")

# We simulate the effect of B by transforming a set of grid points
# Transform each point by matrix B
transformed_grid_B <- as.data.frame(t(B %*% rbind(grid$x, grid$y)))

# Plotting the effect of A on points
p2 <- ggplot() +
  geom_point(aes(x, y), data = grid, color = "blue", alpha = 0.5) +
  geom_point(aes(V1, V2), data = transformed_grid_B, color = "red", alpha = 0.5) +
  xlim(-5, 5) + ylim(-5, 5) +
  labs(title = "Effect of B on Points") +
  theme_minimal()

# Plot transformed A's eigenvectors by B
transformed_vectors <- B %*% eigen_result$vectors
p3 <- plot_vectors(transformed_vectors, "Transforming A's Eigenvectors by B")

# Arrange the plots side by side
grid.arrange(p1, p2, p3, ncol = 3)

Summary

Eigenvalues and eigenvectors provide profound insights into the properties of linear transformations represented by matrices. They reveal the invariant directions under these transformations and the scaling factors associated with those directions.