Table of Contents
1 Introduction to Vectors 1
1.1 VectorsandLinearCombinations...................... 2
1.2 LengthsandDotProducts.......................... 11
1.3 Matrices ................................... 22
2 Solving Linear Equations 31
2.1 VectorsandLinearEquations........................ 31
2.2 TheIdeaofElimination........................... 46
2.3 EliminationUsingMatrices......................... 58
2.4 RulesforMatrixOperations ........................ 70
2.5 InverseMatrices............................... 83
2.6 Elimination = Factorization: A = LU .................. 97
2.7 TransposesandPermutations ........................ 108
3 Vector Spaces and Subspaces 122
3.1 SpacesofVectors .............................. 122
3.2 The Nullspace of A: Solving Ax = 0and Rx =0 ........... 134
3.3 The Complete Solution to Ax = b ..................... 149
3.4 Independence,BasisandDimension .................... 163
3.5 DimensionsoftheFourSubspaces ..................... 180
4 Orthogonality 193
4.1 OrthogonalityoftheFourSubspaces . . . . . . . . . . . . . . . . . . . . 193
4.2 Projections ................................. 205
4.3 LeastSquaresApproximations ....................... 218
4.4 OrthonormalBasesandGram-Schmidt. . . . . . . . . . . . . . . . . . . 232
5 Determinants 246
5.1 ThePropertiesofDeterminants....................... 246
5.2 PermutationsandCofactors......................... 257
5.3 Cramer’sRule,Inverses,andVolumes . . . . . . . . . . . . . . . . . . . 272
vii
6 Eigenvalues and Eigenvectors 287
6.1 IntroductiontoEigenvalues......................... 287
6.2 DiagonalizingaMatrix ........................... 303
6.3 SystemsofDifferentialEquations ..................... 318
6.4 SymmetricMatrices............................. 337
6.5 PositiveDe.niteMatrices.......................... 349
7 TheSingularValueDecomposition (SVD) 363
7.1 ImageProcessingbyLinearAlgebra .................... 363
7.2 BasesandMatricesintheSVD ....................... 370
7.3 Principal Component Analysis (PCA by the SVD) . . . . . . . . . . . . . 381
7.4 TheGeometryoftheSVD ......................... 391
8 LinearTransformations 400
8.1 TheIdeaofaLinearTransformation .................... 400
8.2 TheMatrixofaLinearTransformation. . . . . . . . . . . . . . . . . . . 410
8.3 TheSearchforaGoodBasis ........................ 420
9 ComplexVectorsand Matrices 429
9.1 ComplexNumbers ............................. 430
9.2 HermitianandUnitaryMatrices ...................... 437
9.3 TheFastFourierTransform......................... 444
10 Applications 451
10.1GraphsandNetworks ............................ 451
10.2MatricesinEngineering........................... 461
10.3 Markov Matrices, Population, and Economics . . . . . . . . . . . . . . . 473
10.4LinearProgramming ............................ 482
10.5 Fourier Series: Linear Algebra for Functions . . . . . . . . . . . . . . . . 489
10.6ComputerGraphics ............................. 495
10.7LinearAlgebraforCryptography...................... 501
11 NumericalLinear Algebra 507
11.1GaussianEliminationinPractice ...................... 507
11.2NormsandConditionNumbers....................... 517
11.3 IterativeMethodsandPreconditioners . . . . . . . . . . . . . . . . . . . 523
12LinearAlgebrain Probability& Statistics 534
12.1Mean,Variance,andProbability ...................... 534
12.2 Covariance Matrices and Joint Probabilities . . . . . . . . . . . . . . . . 545
12.3 Multivariate Gaussian and Weighted Least Squares . . . . . . . . . . . . 554
MatrixFactorizations 562
Index 564
SixGreatTheorems/LinearAlgebrain aNutshell 573