0Pricing
R Academy · Lesson

SVD, QR, and Cholesky Decompositions

Apply svd(), qr(), and chol() for dimensionality reduction and factorization.

SVD, QR, and Cholesky Decompositions is a free R Academy lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the R Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Matrix Decompositions?

Matrix decompositions factor A into products of simpler matrices. They reveal hidden structure (rank, condition), enable efficient computation, and are the backbone of PCA, regression, and optimization in statistics and machine learning.

# Three fundamental decompositions:
# 1. SVD: A = U D V'  (any matrix)
# 2. QR:  A = Q R    (any matrix)
# 3. Cholesky: A = L L'  (symmetric positive definite)

# Each serves a purpose:
# SVD -> PCA, pseudoinverse, rank, image compression
# QR  -> linear regression, Gram-Schmidt, eigenvalues
# Cholesky -> fast solve for SPD systems, simulation

A <- matrix(c(4, 3, 2,
              3, 6, 1,
              2, 1, 5), nrow = 3, byrow = TRUE)
cat('Matrix ready for decomposition')
print(A)

SVD: svd() Function

The Singular Value Decomposition (SVD) factorizes any m×n matrix A = U D V', where U and V are orthogonal and D is diagonal with non-negative singular values in decreasing order.

A <- matrix(c(1, 2, 3,
              4, 5, 6), nrow = 2, byrow = TRUE)

# Compute SVD
svd_result <- svd(A)

# Components:
svd_result$d  # singular values (decreasing)
svd_result$u  # left singular vectors  (2x2)
svd_result$v  # right singular vectors (3x2)

cat('Singular values:', svd_result$d, '\n')
cat('Rank of A:', sum(svd_result$d > 1e-10), '\n')

SVD Reconstruction

Reconstruct A from SVD as A = U %*% diag(d) %*% t(V). Low-rank approximations keep only the top k singular values/vectors — this is the basis of image compression and noise reduction.

A <- matrix(c(1, 2, 3,
              4, 5, 6,
              7, 8, 9), nrow = 3, byrow = TRUE)

res <- svd(A)
U <- res$u; d <- res$d; V <- res$v

# Full reconstruction: A = U diag(d) V'
A_reconstructed <- U %*% diag(d) %*% t(V)
all.equal(A, A_reconstructed)  # TRUE

# Rank-1 approximation (best rank-1 approx by Eckart-Young)
A_rank1 <- d[1] * U[, 1] %*% t(V[, 1])
print(round(A_rank1, 2))

# Frobenius error of rank-1 approx
sqrt(sum((A - A_rank1)^2))

SVD: Pseudoinverse and Rank

The SVD gives the pseudoinverse A⁺ = V D⁺ U', where D⁺ replaces each nonzero singular value with its reciprocal. It solves Ax = b in the least-squares sense even when A is not square or full rank.

A <- matrix(c(1, 2, 3,
              4, 5, 6), nrow = 2, byrow = TRUE)

# Pseudoinverse via SVD
svd_res <- svd(A)
tol <- 1e-10
# Invert nonzero singular values
d_inv <- ifelse(svd_res$d > tol, 1/svd_res$d, 0)
A_pinv <- svd_res$v %*% diag(d_inv) %*% t(svd_res$u)
print(A_pinv)  # 3x2 pseudoinverse

# Least-squares solution: x = A+ b
b <- c(7, 8)
x_ls <- A_pinv %*% b
print(x_ls)

# Compare with MASS::ginv
# MASS::ginv(A) should give the same result

QR Decomposition: qr()

QR decomposition factors A = QR where Q is orthogonal (Q'Q = I) and R is upper triangular. It's used in linear regression, computing eigenvalues (QR algorithm), and Gram-Schmidt orthogonalization.

A <- matrix(c(1, 2,
              3, 4,
              5, 6), nrow = 3, byrow = TRUE)

# Compute QR decomposition
qr_res <- qr(A)

# Extract Q and R
Q <- qr.Q(qr_res)  # 3x2 orthogonal
R <- qr.R(qr_res)  # 2x2 upper triangular

cat('Q is orthogonal: Q\'Q = I\n')
print(round(t(Q) %*% Q, 10))  # Identity

cat('R is upper triangular:\n')
print(round(R, 6))

# Reconstruct A = Q %*% R
all.equal(A, Q %*% R)  # TRUE

QR for Regression

Solving the linear regression normal equations via QR is numerically superior. Since Q'Q = I, the normal equations A'Ax = A'b become R'Q'QRx = R'Q'b, simplifying to Rx = Q'b, solved by back-substitution.

# Generate regression data
set.seed(7)
n <- 20
x1 <- rnorm(n); x2 <- rnorm(n)
y <- 1 + 2*x1 - 0.5*x2 + rnorm(n, sd = 0.5)

# Design matrix
X <- cbind(intercept = 1, x1 = x1, x2 = x2)

# QR solve: numerically stable
beta_qr <- qr.solve(X, y)
print(beta_qr)  # ~c(1, 2, -0.5)

# Compare with lm()
beta_lm <- coef(lm(y ~ x1 + x2))
all.equal(beta_qr, beta_lm, check.names = FALSE)  # TRUE

# The qr.solve internals use backsolve on R
cat('QR regression is lm()\'s default method')

Cholesky Decomposition: chol()

Cholesky factorizes a symmetric positive definite matrix A = L L', where L is lower triangular. R's chol() returns the upper triangular factor U (so A = t(U) %*% U).

# Symmetric positive definite matrix
A <- matrix(c(4, 2,
              2, 3), nrow = 2, byrow = TRUE)

# Cholesky decomposition: A = t(U) %*% U
U <- chol(A)
print(U)
# Note: chol() returns UPPER triangular in R

# Reconstruct
all.equal(A, t(U) %*% U)  # TRUE

# Attempting Cholesky on non-PD matrix throws error
# Catch it:
result <- tryCatch(
  chol(matrix(c(1, 3, 3, 2), nrow = 2)),
  error = function(e) cat('Error:', e$message)
)

Solving with Cholesky

For symmetric positive definite systems (like covariance matrices), Cholesky-based solving is twice as fast as LU. Use chol2inv() to compute the inverse, or use backsolve/forwardsolve with the Cholesky factor.

# SPD system: typical in statistics (covariance matrices)
set.seed(1)
X <- matrix(rnorm(30), nrow = 10, ncol = 3)
A <- crossprod(X)  # X'X is always SPD
b <- c(1, 2, 3)

# Cholesky factor
U <- chol(A)  # Upper triangular

# Solve A x = b using two triangular solves
# A = t(U) %*% U -> solve t(U) y = b, then U x = y
y <- forwardsolve(t(U), b)
x <- backsolve(U, y)
print(x)

# Verify
all.equal(as.vector(A %*% x), b)

# Or: inverse via chol2inv
A_inv <- chol2inv(U)
all.equal(x, as.vector(A_inv %*% b))

SVD in PCA

PCA via SVD: center the data matrix X, then compute SVD of centered X. The right singular vectors V are the principal component directions; singular values d relate to variance explained (d²/(n-1)).

set.seed(42)
n <- 50; p <- 3
X <- matrix(rnorm(n * p), nrow = n)
# Add correlation
X[, 2] <- 0.8 * X[, 1] + 0.6 * X[, 2]

# Center
X_c <- scale(X, center = TRUE, scale = FALSE)

# SVD of centered data
svd_res <- svd(X_c)

# Variance explained by each PC
var_explained <- svd_res$d^2 / (n - 1)
prop_var <- var_explained / sum(var_explained)
cat('Proportion of variance explained:\n')
print(round(prop_var, 3))

# Compare with prcomp
pc <- prcomp(X)
all.equal(prop_var, pc$sdev^2 / sum(pc$sdev^2))

Matrix Rank via SVD

The rank of a matrix equals the number of non-negligible singular values. SVD provides the most numerically reliable way to determine rank — simply count d values above a threshold (typically max(d) * tol).

# Full-rank matrix (rank 3)
A <- matrix(c(1, 0, 0,
              0, 2, 0,
              0, 0, 3), nrow = 3)
svd(A)$d  # c(3, 2, 1) - all nonzero -> rank 3

# Rank-deficient matrix (rank 2)
B <- matrix(c(1, 2, 3,
              2, 4, 6,   # row 2 = 2 * row 1
              0, 1, 1), nrow = 3, byrow = TRUE)
svd(B)$d  # Third value is ~0

# Rank function using SVD
matrix_rank <- function(M, tol = 1e-10) {
  d <- svd(M)$d
  sum(d > tol * d[1])  # relative tolerance
}
matrix_rank(B)  # 2

Cholesky for Sampling Multivariate Normal

To sample from a multivariate normal N(μ, Σ), compute Cholesky factor U of Σ, then transform standard normals: X = μ + Z %*% U. This is how MASS::mvrnorm() works internally.

set.seed(99)
# Covariance matrix
Sigma <- matrix(c(4, 2,
                  2, 3), nrow = 2)
mu <- c(1, 5)
n_samples <- 500

# Cholesky factor
U <- chol(Sigma)

# Sample standard normals
Z <- matrix(rnorm(n_samples * 2), nrow = n_samples)

# Transform: X ~ N(mu, Sigma)
X <- sweep(Z %*% U, 2, mu, FUN = '+')

# Check empirical covariance
print(round(cov(X), 2))  # Should be close to Sigma
print(round(colMeans(X), 2))  # Should be close to mu

Quick Check

Test your understanding of SVD, QR, and Cholesky decompositions in R.

Recap: SVD, QR, and Cholesky

Key takeaways: SVD (svd()) returns d, u, v — used for PCA, rank, pseudoinverse, and low-rank approximation. QR (qr()) returns an orthogonal Q and upper triangular R — used for regression and stability. Cholesky (chol()) returns upper triangular U where A = t(U)%*%U — fastest for SPD systems. All reconstruct A exactly and serve different computational purposes.

A <- matrix(c(4, 2, 2, 3), nrow = 2)

# SVD
res <- svd(A)
all.equal(A, res$u %*% diag(res$d) %*% t(res$v))  # TRUE

# QR (on a design matrix)
X <- cbind(1, c(1,2,3,4,5))
qr_res <- qr(X)
all.equal(X, qr.Q(qr_res) %*% qr.R(qr_res))  # TRUE

# Cholesky
U <- chol(A)
all.equal(A, t(U) %*% U)  # TRUE

# Use cases:
# svd() -> PCA, rank, pseudoinverse
# qr()  -> regression, stability
# chol()-> SPD solve, MVN sampling

Frequently asked questions

Is the “SVD, QR, and Cholesky Decompositions” lesson free?

Yes — the full text of “SVD, QR, and Cholesky Decompositions” is free to read here on the web, and the R Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the R Academy course, upgrade to CoddyKit PRO.

What will I learn in “SVD, QR, and Cholesky Decompositions”?

Apply svd(), qr(), and chol() for dimensionality reduction and factorization. You practise R Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start R Academy?

No prior experience is required. R Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “SVD, QR, and Cholesky Decompositions” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this R Academy lesson?

Yes. Every R Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Matrix Multiplication and Determinants
  2. Solving Linear Systems with solve()
  3. Eigenvalues and Eigenvectors
  4. SVD, QR, and Cholesky Decompositions
← Back to R Academy