Eigenvalues and SVD Overview
Compute eigenvalues and eigenvectors with np.linalg.eig and understand how SVD underpins PCA dimensionality reduction.
What Are Eigenvalues and Eigenvectors?
An eigenvector of a square matrix A is a non-zero vector v such that A @ v = lambda * v — multiplying by A only scales v, it does not change its direction. The scalar lambda is called the eigenvalue corresponding to v. Eigenvalues reveal the intrinsic 'stretching factors' of a linear transformation: an eigenvalue of 2 means the matrix doubles lengths in the eigenvector direction; a negative eigenvalue flips direction.
import numpy as np
A = np.array([[3.0, 1.0],
[0.0, 2.0]])
eigenvalues, eigenvectors = np.linalg.eig(A)
print('Eigenvalues:', eigenvalues)
print('Eigenvectors (columns):')
print(eigenvectors)Computing Eigenvalues with np.linalg.eig()
np.linalg.eig(A) returns a tuple (eigenvalues, eigenvectors). The eigenvalues are a 1-D array; the eigenvectors are a 2-D array where each column is an eigenvector. For real symmetric matrices (like covariance matrices), the eigenvalues are always real and the eigenvectors are orthogonal — use np.linalg.eigh(A) for symmetric matrices as it is faster and guaranteed to return real results.
import numpy as np
# Symmetric matrix -> use eigh for efficiency and real eigenvalues
A = np.array([[4.0, 2.0],
[2.0, 3.0]])
vals, vecs = np.linalg.eigh(A)
print('Eigenvalues (real):', vals)
print('Eigenvectors (orthonormal columns):')
print(vecs)
# Verify: A @ v = lambda * v for each eigenvector
for i in range(len(vals)):
lhs = A @ vecs[:, i]
rhs = vals[i] * vecs[:, i]
print(f'v{i} check:', np.allclose(lhs, rhs))All lessons in this course
- Matrix Multiplication with np.matmul and @
- Determinants, Inverses, and Transposes
- Solving Linear Systems
- Eigenvalues and SVD Overview