Contents
1 Characteristics of Time Series 1
1.1 Introduction 1
1.2 The Nature of Time Series Data 3
1.3 Time Series Statistical Models 11
1.4 Measures of Dependence: Autocorrelation and Cross-Correlation 17
1.5 Stationary Time Series 22
1.6 Estimation of Correlation 28
1.7 Vector-Valued and Multidimensional Series 33
2 Time Series Regression and Exploratory Data Analysis 47
2.1 Introduction 47
2.2 Classical Regression in the Time Series Context 48
2.3 Exploratory Data Analysis 57
2.4 Smoothing in the Time Series Context 70
3 ARIMA Models 83
3.1 Introduction 83
3.2 Autoregressive Moving Average Models 84
3.3 Difference Equations 97
3.4 Autocorrelation and Partial Autocorrelation 102
3.5 Forecasting 108
3.6 Estimation 121
3.7 Integrated Models for Nonstationary Data 141
3.8 Building ARIMA Models 144
3.9 Multiplicative Seasonal ARIMA Models 154
4 Spectral Analysis and Filtering 173
4.1 Introduction 173
4.2 Cyclical Behavior and Periodicity 175
4.3 The Spectral Density 180
4.4 Periodogram and Discrete Fourier Transform 187
4.5 Nonparametric Spectral Estimation 196
4.6 Parametric Spectral Estimation 212
4.7 Multiple Series and Cross-Spectra 216
4.8 Linear Filters 221
4.9 Dynamic Fourier Analysis and Wavelets 228
4.10 Lagged Regression Models 242
4.11 Signal Extraction and Optimum Filtering 247
4.12 Spectral Analysis of Multidimensional Series 252
5 Additional Time Domain Topics 267
5.1 Introduction 267
5.2 Long Memory ARMA and Fractional Differencing 267
5.3 Unit Root Testing 277
5.4 GARCH Models 280
5.5 Threshold Models 289
5.6 Regression with Autocorrelated Errors 293
5.7 Lagged Regression: Transfer Function Modeling 296
5.8 Multivariate ARMAX Models 301
6 State-Space Models 319
6.1 Introduction 319
6.2 Filtering, Smoothing, and Forecasting 325
6.3 Maximum Likelihood Estimation 335
6.4 Missing Data Modifications 344
6.5 Structural Models: Signal Extraction and Forecasting 350
6.6 State-Space Models with Correlated Errors 354
6.6.1 ARMAX Models 355
6.6.2 Multivariate Regression with Autocorrelated Errors 356
6.7 Bootstrapping State-Space Models 359
6.8 Dynamic Linear Models with Switching 365
6.9 Stochastic Volatility 378
6.10 Nonlinear and Non-normal State-Space Models Using Monte Carlo Methods 387
7 Statistical Methods in the Frequency Domain 405
7.1 Introduction 405
7.2 Spectral Matrices and Likelihood Functions 409
7.3 Regression for Jointly Stationary Series 410
7.4 Regression with Deterministic Inputs 420
7.5 Random Coefficient Regression 429
7.6 Analysis of Designed Experiments 434
7.7 Discrimination and Cluster Analysis 450
7.8 Principal Components and Factor Analysis 468
7.9 The Spectral Envelope 485
Appendix A: Large Sample Theory 507
A.1 Convergence Modes 507
A.2 Central Limit Theorems 515
A.3 The Mean and Autocorrelation Functions 518
Appendix B: Time Domain Theory 527
B.1 Hilbert Spaces and the Projection Theorem 527
B.2 Causal Conditions for ARMA Models 531
B.3 Large Sample Distribution of the AR(p) Conditional Least Squares Estimators 533
B.4 The Wold Decomposition 537
Appendix C: Spectral Domain Theory 539
C.1 Spectral Representation Theorem 539
C.2 Large Sample Distribution of the DFT and Smoothed Periodogram 543
C.3 The Complex Multivariate Normal Distribution 554
Appendix R: R Supplement 559
R.1 First Things First 559
R.1.1 Included Data Sets 560
R.1.2 Included Scripts 562
R.2 Getting Started 567
R.3 Time Series Primer 571