I Vectors
1 Vectors ix
1.1 Vectors
1.2 Vector addition
1.3 Scalar-vector multiplication
1.4 Inner product
1.5 Complexity of vector computations
Exercises
2 Linear functions
2.1 Linear functions
2.2 Taylor approximation
2.3 Regression model
Exercises
3 Norm and distance
3.1 Norm
3.2 Distance
3.3 Standard deviation
3.4 Angle
3.5 Complexity
Exercises
4 Clustering
4.1 Clustering
4.2 A clustering objective
4.3 The k-means algorithm
4.4 Examples
4.5 Applications
Exercises
5 Linear independence
5.1 Linear dependence
5.2 Basis
5.3 Orthonormal vectors
5.4 Gram–Schmidt algorithm
Exercises
II Matrices
6 Matrices
6.1 Matrices
6.2 Zero and identity matrices
6.3 Transpose, addition, and norm
6.4 Matrix-vector multiplication
6.5 Complexity
Exercises
7 Matrix examples
7.1 Geometric transformations
7.2 Selectors
7.3 Incidence matrix
7.4 Convolution
Exercises
8 Linear equations
8.1 Linear and affine functions
8.2 Linear function models
8.3 Systems of linear equations
Exercises
9 Linear dynamical systems
9.1 Linear dynamical systems
9.2 Population dynamics
9.3 Epidemic dynamics
9.4 Motion of a mass
9.5 Supply chain dynamics
Exercises
10 Matrix multiplication
10.1 Matrix-matrix multiplication
10.2 Composition of linear functions
10.3 Matrix power
10.4 QR factorization
Exercises
11 Matrix inverses
11.1 Left and right inverses
11.2 Inverse
11.3 Solving linear equations
11.4 Examples
11.5 Pseudo-inverse
Exercises
III Least squares
12 Least squares
12.1 Least squares problem
12.2 Solution
12.3 Solving least squares problems
12.4 Examples
Exercises
13 Least squares data fitting
13.1 Least squares data fitting
13.2 Validation
13.3 Feature engineering
Exercises
14 Least squares classification
14.1 Classification
14.2 Least squares classifier
14.3 Multi-class classifiers
Exercises
15 Multi-objective least squares
15.1 Multi-objective least squares
15.2 Control
15.3 Estimation and inversion
15.4 Regularized data fitting
15.5 Complexity
Exercises
16 Constrained least squares
16.1 Constrained least squares problem
16.2 Solution
16.3 Solving constrained least squares problems
Exercises
17 Constrained least squares applications
17.1 Portfolio optimization
17.2 Linear quadratic control
17.3 Linear quadratic state estimation
Exercises
18 Nonlinear least squares
18.1 Nonlinear equations and least squares
18.2 Gauss–Newton algorithm
18.3 Levenberg–Marquardt algorithm
18.4 Nonlinear model fitting
18.5 Nonlinear least squares classification
Exercises
19 Constrained nonlinear least squares
19.1 Constrained nonlinear least squares
19.2 Penalty algorithm
19.3 Augmented Lagrangian algorithm
19.4 Nonlinear control
Exercises
Appendices
A Notation
B Complexity
C Derivatives and optimization
C.1 Derivatives
C.2 Optimization
C.3 Lagrange multipliers
D Further study Index