intuition.
  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
Linear Algebra
Step 21 of 22

Symmetric matrices

Understand first: diagonalization

Diagonalization came with a warning: not every matrix has enough independent eigenvectors. One family escapes that worry entirely, and it is not an obscure special case — it is the family that shows up when you measure real data.

A matrix is symmetric when it equals its own transpose: flip it across the main diagonal and nothing changes. Aij=AjiA_{ij} = A_{ji}.

A=[2112]symmetric,B=[2102]notA = \begin{bmatrix} 2 & 1 \ 1 & 2 \end{bmatrix} \quad \text{symmetric}, \qquad B = \begin{bmatrix} 2 & 1 \ 0 & 2 \end{bmatrix} \quad \text{not}

The guarantee

For any symmetric matrix, with no exceptions:

  1. Every eigenvalue is a real number, never complex.
  2. Eigenvectors from different eigenvalues are perpendicular to each other.
  3. There are always enough of them to form a basis, so it is always diagonalizable.

Check the second claim on our example. Its eigenvectors were (1,1)(1,1) and (1,−1)(1,-1), so take the dot product from step 16:

(1,1)⋅(1,−1)=1(1)+1(−1)=0(1,1) \cdot (1,-1) = 1(1) + 1(-1) = 0

Zero, so perpendicular. Watch the visual: as the matrix changes, the amber axes stretch and swing, but the right angle between them never breaks.

What that means geometrically

Put it together. A symmetric matrix is diagonalizable in a basis of perpendicular axes. So its action is: find a set of right-angled directions, stretch along each by its eigenvalue, done.

No shear. No rotation. No twisting of any kind. It is the cleanest possible motion, and the perpendicular frame means the new basis behaves like the original one — angles and lengths measured in it still work normally.

Why this is the last brick

Symmetry is not a rare coincidence. Any matrix built by comparing every variable against every other variable is automatically symmetric, because comparing ii to jj gives the same answer as comparing jj to ii.

That is precisely how a covariance matrix is built from data.

So a covariance matrix is guaranteed to have perpendicular eigenvectors and real eigenvalues. Everything is now in place for the destination this path has been climbing toward since the first arrow.