Eigenvalues

An eigenvalue is the scalar by which the eigenvector is scaled. Mathematically, eigenvalues are defined as:

For a square matrix A, a scalar, and a non-zero column vector v to satisfy the below-mentioned condition, 
                 \(A\vec v = \lambda \vec v\)

Then,
\(\vec v\) must be an eigenvector of A
λ must be an eigenvalue of A.

A comprehensive understanding of the properties of eigenvalues is fundamental for accurate interpretation of linear transformations and facilitation of matrix operations.

• A square matrix of order \(n × n\) can have a maximum of n eigenvalues.
   
• All n eigenvalues of an identity matrix are equal to 1.
   
• For both triangular and diagonal matrices, the eigenvalues are the elements present on the main diagonal.
   
• The sum of the eigenvalues of a matrix is equal to the sum of its diagonal elements. When the eigenvalues are multiplied together, we get the determinant.
   
• Hermitian and symmetric matrices have real eigenvalues.
   
• The eigenvalues of skew-Hermitian and skew-symmetric matrices are restricted to purely imaginary values or zeroes.
   
• A matrix and its transpose have the same eigenvalues.
   
• Consider two square matrices A and B. If they are of the same order, then AB and BA have the same, non-zero eigenvalues. However, their zero eigenvalues might differ.
   
• An orthogonal matrix’s eigenvalues have an absolute value of 1. They can be real, i.e., 1/-1 or complex conjugate pairs.
   
• For any scalar k, the eigenvalues of the matrix kA are obtained by multiplying each eigenvalue of matrix A by K.
   
• If λ is an eigenvalue of matrix A, then λk is an eigenvalue of Ak, provided that A is diagonalizable.
   
• For an invertible matrix A, each eigenvalue becomes 1 for the inverse of the matrix A-1.
   
• If λ is a non-zero eigenvalue of A, then |A| / λ is an eigenvalue of the adjoint of A.

It is also important to understand the Cayley-Hamilton Theorem, which states that every square matrix satisfies its own characteristic equation.

For a characteristic polynomial of A:

\(p(λ)=det(A-λI)=λⁿ+a₁λⁿ⁻¹+...+aₙ₋₁λ+aₙ\)

Then the Cayley-Hamilton Theorem states:

\( p(A) = A^n + a_{1}A^{\,n-1} + \cdots + a_{n-1}A + a_{n}I = 0\)

As the properties suggest, if λ is an eigenvalue for given square matrix A, then
\(Av = λv\)
 

If identity matrix I and matrix A are of the same order, then:

\(Av = λ(Iv) (v = Iv)\)

\(Av -  λ(Iv) = 0\)

v is the common factor, so,

v(A - λI) = 0

This is a homogeneous system. The existence of \(v \ne 0\) implies that \(det(A - λI) = 0\). This is the characteristic equation.

Here, \(det(A - λI)\)  is known as the characteristic polynomial and λ is the eigenvalue.

To find eigenvalues of a square matrix:

Step 1: Consider a square matrix A.

Step 2: Let I be the identity matrix of the same order as A.

Step 3: Subtract  λI from A.

Step 4: Find the determinant.

Step 5: Equate determinant = 0 and find the value of λ.

Using the steps mentioned above, let's solve an example:
Let's take the matrix:                           

\(A = \begin{bmatrix} 3 & 2 \\ 2 & 3 \end{bmatrix}\)

Let λ represent the eigenvalues.
Identity matrix I:
\(I = \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} \implies \lambda I = \begin{bmatrix} \lambda & 0 \\ 0 & \lambda \end{bmatrix}\)

\(A - \lambda I = \begin{bmatrix} 3 - \lambda & 2 \\ 2 & 3 - \lambda \end{bmatrix}\)                         
Finding the determinant: 

\( \begin{align*} |A - \lambda I| &= (3 - \lambda)(3 - \lambda) - (2)(2) \\ &= (3 - \lambda)^2 - 4 \\ &= 9 - 6\lambda + \lambda^2 - 4 \\ &= \lambda^2 - 6\lambda + 5 \end{align*}\)
 

Characteristic equation:

                 \(λ² - 6λ + 5 = 0\)                

          
Factoring it, we get:

                  \(\begin{align*} (\lambda - 5)(\lambda - 1) &= 0 \\ \lambda = 5, \lambda &= 1 \end{align*}\)             
                   

The eigenvalues for the given matrix are 5 and 1.

In this section, we will use the steps mentioned in the previous segment to find the eigenvalues of a 3 × 3 matrix. Let’s consider the following matrix:    

\(A = \begin{bmatrix} 2 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 5 \end{bmatrix}\)                      

Characteristic equation \(|A - \lambda I = 0|\)

Subtracting from each diagonal entry, we get: 

\(A - \lambda I = \begin{bmatrix} 2-\lambda & 0 & 0 \\ 0 & 3-\lambda & 0 \\ 0 & 0 & 5-\lambda \end{bmatrix}\)
                       
Now, determinant:

\( det ( A - \lambda I ) = ( 2 - \lambda ) ( 3 - \lambda ) ( 5 - \lambda)\)

Solving for: \((2 - \lambda) (3 - \lambda ) ( 5 - \lambda) = 0 \)

 \(\lambda = 2, \, 3, \, 5\)

The eigenvalues of matrix A are: 

  \(\lambda = 2, \, 3, \, 5\)

Since the matrix is diagonal, the eigenvalues are the diagonal entries: λ = 2, 3, 5.

Note: The matrix here is diagonal for simplicity, which makes the eigenvalues directly equal to its diagonal entries.

Eigenvalues play a key role in understanding the structure and behavior of various systems across fields like engineering, physics, and data science. Below are some important real-world applications of eigenvalues:

• Mechanical Vibrations: In engineering, eigenvalues determine natural frequencies of systems, crucial in engineering designs
  
   
• Principal Component Analysis (PCA): In data science, eigenvalues help identify principal components for dimensionality reduction
  
   
• Quantum Mechanics: In quantum mechanics, eigenvalues correspond to observable quantities such as energy levels of particles, helping describe their physical states.
  
   
• Stability Analysis: In control theory, eigenvalues are used to assess system stability. If the eigenvalues have certain properties, the system is stable and behaves predictably.
  
   
• Facial Recognition: The Eigenfaces technique is a popular method in facial recognition that uses eigenvalues and eigenvectors for image recognition.