Determinants, Inverses, and Transposes

The transpose of a matrix flips it over its diagonal: rows become columns and columns become rows. If A has shape (m, n), then A.T has shape (n, m). Transposes appear everywhere in linear algebra: computing covariance matrices, implementing gradient backpropagation, and converting between row-vector and column-vector conventions. NumPy accesses the transpose with the .T attribute — no copy is made, it is just a view with reordered strides.

A frequent pattern is computing A.T @ A, which produces a symmetric square matrix that appears in least squares regression, PCA, and normal equations. If A has shape (n, p), then A.T @ A has shape (p, p). The result is always symmetric because (A.T @ A)[i,j] == (A.T @ A)[j,i]. This property is exploited by many numerical solvers for efficiency.

The determinant of a square matrix is a scalar value that encodes geometric information: it measures the factor by which the matrix scales areas (2-D) or volumes (higher dimensions). A determinant of zero means the matrix is singular — it collapses space onto a lower-dimensional subspace and has no inverse. A non-zero determinant guarantees the matrix is invertible. NumPy computes it with np.linalg.det().

The inverse of a matrix A, written A⁻¹, satisfies A @ A⁻¹ = I where I is the identity matrix. Inverses exist only for square, non-singular matrices. np.linalg.inv(A) computes the inverse numerically. In practice, you should rarely compute inverses explicitly because it is numerically unstable; prefer np.linalg.solve() for systems of equations instead. But understanding inverses conceptually is essential.

Computing the inverse of a matrix and then multiplying introduces floating-point errors that compound. For example, solving A_inv @ b is less accurate than calling np.linalg.solve(A, b) directly. A well-conditioned matrix (determinant far from zero) is less sensitive to these errors. The condition number (np.linalg.cond(A)) quantifies how close a matrix is to singular — high condition numbers signal numerical trouble.

A special class of matrices is orthogonal matrices, where A.T @ A = I. This means the transpose IS the inverse, making orthogonal matrices extremely cheap to invert. Rotation matrices and the Q factor in QR decomposition are orthogonal. The columns of an orthogonal matrix form an orthonormal basis — each column has unit length and is perpendicular to every other column.

A regular inverse only exists for square matrices, but the Moore-Penrose pseudo-inverse generalises inversion to any matrix. np.linalg.pinv(A) computes it using SVD. It is the basis of the least-squares solution to over-determined systems (more equations than unknowns), which is exactly what linear regression solves. If A is square and invertible, pinv(A) equals inv(A).

The sign of the determinant indicates whether the transformation preserves or reverses orientation. A positive determinant means orientation is preserved (like a rotation); a negative determinant means orientation is flipped (like a reflection). The absolute value gives the volume scaling factor. For a 2×2 matrix [[a,b],[c,d]], det = a*d - b*c. NumPy handles arbitrary square sizes efficiently.

For large matrices, the determinant can overflow or underflow to zero even when the matrix is non-singular. np.linalg.slogdet(A) returns (sign, log_abs_det) — the sign and the natural log of the absolute determinant — which avoids numerical overflow. This is widely used in probabilistic models (Gaussian likelihoods) where you work in log-space to keep numbers manageable.

For a 2×2 matrix [[a, b], [c, d]], the inverse is (1/det) * [[d, -b], [-c, a]] where det = ad - bc. Knowing this formula helps you spot-check NumPy's result. For anything larger, computing the inverse analytically is impractical — that is precisely why we use np.linalg.inv(). Always verify numerical results with np.allclose(A @ A_inv, np.eye(n)).

A common linear algebra pattern combines transpose and inverse: computing (A.T @ A)⁻¹ @ A.T gives the projection matrix used in ordinary least squares. In NumPy, this can be written as np.linalg.inv(A.T @ A) @ A.T but again, np.linalg.lstsq is numerically preferable. Knowing the underlying pattern helps you understand what the solver is doing and debug issues when matrices are near-singular.

Test your understanding of Data Analysis concepts from this lesson.

In this lesson you learned: A.T is the transpose that swaps rows and columns, np.linalg.inv() computes the matrix inverse but should rarely be used directly (prefer solve or lstsq), and np.linalg.det() returns the determinant which is zero for singular matrices. Next up we explore solving linear systems directly with np.linalg.solve.