TSM 2025-2026 Full Study Guide 1
Time Series Models: Full Study Guide
Weeks 1–6
Contents
I Week 1: Local Level Model & Signal Extraction 2
1 The Big Picture: How Everything Connects 2
2 From Regression to State Space 2
2.1 Signal + Noise: The Regression Perspective . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Why the LRM Fails for the Nile Data . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.3 Deterministic vs. Stochastic Time-Varying Level . . . . . . . . . . . . . . . . . . . . 3
2.4 Course Outlook: Three Model Classes . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 The Local Level Model 3
3.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2 Signal-to-Noise Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 Properties: LLM as ARIMA(0,1,1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.4 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.5 Unconditional Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4 The Kalman Filter 6
4.1 Theory: Filtering vs. Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 The KF Recursion for the LLM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.3 Building the KF Step by Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.4 Key Insight: Prediction Error Decomposition . . . . . . . . . . . . . . . . . . . . . . 8
4.5 Worked Example 1: Nile Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.6 Worked Example 2: General Model (Exam Style) . . . . . . . . . . . . . . . . . . . . 10
4.7 Performance and Steady State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.8 Weight Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5 The Kalman Smoother 11
5.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.2 The KS Recursion for the LLM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.3 Why Does Future Data Help? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.4 Building the KS Step by Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.5 Comparing KF and KS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.6 KS Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6 Missing Observations and Forecasting 18
6.1 Missing Observations Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.3 Worked Example: Missing Observation with Mixed Initialization (Exam Style) . . . 19
7 Parameter Estimation by Maximum Likelihood 19
7.1 Why the Likelihood Is Directly Visible from the KF . . . . . . . . . . . . . . . . . . 19
7.2 Diffuse Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
7.3 Estimation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
,TSM 2025-2026 Full Study Guide 2
8 Multivariate Normal Conditional Expectation 22
8.1 Lemma I: Conditioning on One Variable . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.2 Lemma II: Sequential Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.3 Connection to the Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.4 Exam Recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.5 Worked Example (2025 MC.9 Format) . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.6 Practice Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9 Signal Extraction: Which Method for Which Goal? 25
II Week 2: Linear Gaussian Models 27
10 The Big Picture: From LLM to Linear Gaussian Models 27
11 The Linear Gaussian Model 27
11.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
11.2 State versus Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
12 Example Models in State Space Form 28
12.1 Example 1: Local Level Model (Recap) . . . . . . . . . . . . . . . . . . . . . . . . . 28
12.2 Example 2: AR(1) + Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
12.3 Example 3: Local Linear Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
12.4 Example 4: Time-Varying Linear Regression . . . . . . . . . . . . . . . . . . . . . . . 30
12.5 Example 5: ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
13 General Kalman Filter 32
13.1 The KF Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
13.2 Initialisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
13.3 Missing Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
13.4 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
13.5 Multivariate Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
14 Proving the Kalman Filter 37
14.1 Lemma I: Conditional Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . 37
14.2 Lemma II: Updating with New Information . . . . . . . . . . . . . . . . . . . . . . . 38
14.3 Proof Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
14.4 Full Proof: General LGM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
14.5 Alternative: Scalar LLM Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
15 General Kalman Smoother 42
15.1 KS Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
15.2 Derivation of the KS Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
15.3 Disturbance Smoother . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
16 Maximum Likelihood and the Diffuse Log-Likelihood 47
16.1 Log-Likelihood via Prediction Error Decomposition . . . . . . . . . . . . . . . . . . . 47
16.2 Diffuse Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
17 Residual Diagnostics 48
17.1 Standardised Residuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
III Week 3: Simulation Smoothing, Nonlinear Gaussian Models & SV 49
,TSM 2025-2026 Full Study Guide 3
18 The Big Picture: From Linear to Nonlinear 49
19 Nonlinear Functions of the State 50
19.1 Linear vs. Nonlinear Signal Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . 50
19.2 The Problem with Nonlinear Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 50
19.3 Monte Carlo Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
19.4 The Smoothed Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
20 Lemma III 53
20.1 Intuition and Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
20.2 Proof of Lemma III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
21 Simulation Smoothing 56
21.1 Applying Lemma III to the State Space Model . . . . . . . . . . . . . . . . . . . . . 57
21.2 Notation Convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
21.3 The Unconditional Simulation Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
21.4 The Full Simulation Smoothing Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 57
22 The Nonlinear Gaussian Model (nLGM) 58
22.1 Forms of Nonlinearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
22.2 Example: UK Visits Abroad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
23 Taylor Expansion and Linearization 59
23.1 Formal Taylor Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
23.2 The Linearized Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
24 Extended Kalman Filter and Smoother 61
24.1 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
25 The Stochastic Volatility Model and QML 63
25.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
25.2 Can We Use the EKF for the SV-Model? . . . . . . . . . . . . . . . . . . . . . . . . 64
25.3 Data Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
25.4 Quasi Maximum Likelihood (QML) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
25.5 The Approximating LGM for QML . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
IV Week 4: Nonlinear Non-Gaussian Models & Mode Estimation 66
26 The Big Picture: From Gaussian to Non-Gaussian 66
27 The Nonlinear Non-Gaussian Model 67
27.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
27.2 Running Example: Boat Race Wins (Bernoulli) . . . . . . . . . . . . . . . . . . . . . 68
27.3 How the Pieces Fit Together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
27.4 Why Is the State Equation Still Gaussian? . . . . . . . . . . . . . . . . . . . . . . . . 70
27.5 The Three Model Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
28 Example Models: The Exponential Family 71
28.1 The Exponential Family of Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 71
28.2 Example 1: Poisson Model (Count Data) . . . . . . . . . . . . . . . . . . . . . . . . 72
28.3 Example 2: Binary/Bernoulli Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
28.4 Example 3: Three Special Cases of nLnGMs . . . . . . . . . . . . . . . . . . . . . . . 74
, TSM 2025-2026 Full Study Guide 4
29 Stacked Form Notation 74
29.1 Starting Point: The Per-Period Model . . . . . . . . . . . . . . . . . . . . . . . . . . 74
29.2 Step 1: Stack the Observations and Signals . . . . . . . . . . . . . . . . . . . . . . . 75
29.3 Step 2: The Signal Equation θ = d + Zα . . . . . . . . . . . . . . . . . . . . . . . . . 75
29.4 Step 3: The State Distribution α ∼ N (c, Γ) . . . . . . . . . . . . . . . . . . . . . . . 75
29.5 Step 4: The Observation Equation and the Complete Stacked Model . . . . . . . . . 76
29.6 Key Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
29.7 LGM Properties in Stacked Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
30 The Inference Pipeline 78
31 Mode Estimation 86
31.1 The Goal: Why Target the Mode? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
31.2 The LGM Mode: A Closed-Form Template . . . . . . . . . . . . . . . . . . . . . . . 87
31.3 Newton-Raphson Meets Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
31.4 The Surrogate LGM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
31.5 The Mode Estimation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
31.6 Two Outputs: The Mode and the Gaussian Approximation . . . . . . . . . . . . . . 97
32 Importance Sampling 98
32.1 Why Importance Sampling? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
32.2 Signal Extraction: From Weighted Samples to Posterior Quantities . . . . . . . . . . 101
32.3 Prediction: Assembling the Time-Domain Output . . . . . . . . . . . . . . . . . . . . 102
32.4 The SPDK Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
32.5 The Four Densities: Known vs Unknown . . . . . . . . . . . . . . . . . . . . . . . . . 103
33 Signal Extraction Methods: The Full Picture 105
33.1 The Eight Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
33.2 Decision Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
33.3 When to Use What: Summary Table . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
V Week 5: Importance Sampling, Parameter Estimation & Bootstrap Filter107
34 Week 4 Recap: Where We Left Off 107
34.1 The nLnGM Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
34.2 The Five-Step Inference Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
34.3 W4 to W5 Bridge: What We Have vs. What We Need . . . . . . . . . . . . . . . . . 108
34.4 What Week 5 Adds and Why It Matters . . . . . . . . . . . . . . . . . . . . . . . . . 108
35 The Big Picture: Completing the Inference Toolbox 108
35.1 The Signal Extraction Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
36 The Importance Sampling Problem 109
36.1 What We Want . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
36.2 Why Direct Computation Fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
37 The SPDK Importance Density 110
37.1 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
37.2 The Six Densities and Their Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
Time Series Models: Full Study Guide
Weeks 1–6
Contents
I Week 1: Local Level Model & Signal Extraction 2
1 The Big Picture: How Everything Connects 2
2 From Regression to State Space 2
2.1 Signal + Noise: The Regression Perspective . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 Why the LRM Fails for the Nile Data . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.3 Deterministic vs. Stochastic Time-Varying Level . . . . . . . . . . . . . . . . . . . . 3
2.4 Course Outlook: Three Model Classes . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3 The Local Level Model 3
3.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
3.2 Signal-to-Noise Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3.3 Properties: LLM as ARIMA(0,1,1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.4 Initialization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3.5 Unconditional Simulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4 The Kalman Filter 6
4.1 Theory: Filtering vs. Prediction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.2 The KF Recursion for the LLM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4.3 Building the KF Step by Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
4.4 Key Insight: Prediction Error Decomposition . . . . . . . . . . . . . . . . . . . . . . 8
4.5 Worked Example 1: Nile Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4.6 Worked Example 2: General Model (Exam Style) . . . . . . . . . . . . . . . . . . . . 10
4.7 Performance and Steady State . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4.8 Weight Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5 The Kalman Smoother 11
5.1 Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5.2 The KS Recursion for the LLM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.3 Why Does Future Data Help? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.4 Building the KS Step by Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.5 Comparing KF and KS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
5.6 KS Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6 Missing Observations and Forecasting 18
6.1 Missing Observations Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
6.2 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
6.3 Worked Example: Missing Observation with Mixed Initialization (Exam Style) . . . 19
7 Parameter Estimation by Maximum Likelihood 19
7.1 Why the Likelihood Is Directly Visible from the KF . . . . . . . . . . . . . . . . . . 19
7.2 Diffuse Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
7.3 Estimation Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
,TSM 2025-2026 Full Study Guide 2
8 Multivariate Normal Conditional Expectation 22
8.1 Lemma I: Conditioning on One Variable . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.2 Lemma II: Sequential Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
8.3 Connection to the Kalman Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.4 Exam Recipe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.5 Worked Example (2025 MC.9 Format) . . . . . . . . . . . . . . . . . . . . . . . . . . 24
8.6 Practice Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
9 Signal Extraction: Which Method for Which Goal? 25
II Week 2: Linear Gaussian Models 27
10 The Big Picture: From LLM to Linear Gaussian Models 27
11 The Linear Gaussian Model 27
11.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
11.2 State versus Signal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
12 Example Models in State Space Form 28
12.1 Example 1: Local Level Model (Recap) . . . . . . . . . . . . . . . . . . . . . . . . . 28
12.2 Example 2: AR(1) + Noise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
12.3 Example 3: Local Linear Trend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
12.4 Example 4: Time-Varying Linear Regression . . . . . . . . . . . . . . . . . . . . . . . 30
12.5 Example 5: ARIMA Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
13 General Kalman Filter 32
13.1 The KF Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
13.2 Initialisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
13.3 Missing Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
13.4 Forecasting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
13.5 Multivariate Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
14 Proving the Kalman Filter 37
14.1 Lemma I: Conditional Mean and Variance . . . . . . . . . . . . . . . . . . . . . . . . 37
14.2 Lemma II: Updating with New Information . . . . . . . . . . . . . . . . . . . . . . . 38
14.3 Proof Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
14.4 Full Proof: General LGM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
14.5 Alternative: Scalar LLM Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
15 General Kalman Smoother 42
15.1 KS Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
15.2 Derivation of the KS Recursion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
15.3 Disturbance Smoother . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
16 Maximum Likelihood and the Diffuse Log-Likelihood 47
16.1 Log-Likelihood via Prediction Error Decomposition . . . . . . . . . . . . . . . . . . . 47
16.2 Diffuse Log-Likelihood . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
17 Residual Diagnostics 48
17.1 Standardised Residuals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
III Week 3: Simulation Smoothing, Nonlinear Gaussian Models & SV 49
,TSM 2025-2026 Full Study Guide 3
18 The Big Picture: From Linear to Nonlinear 49
19 Nonlinear Functions of the State 50
19.1 Linear vs. Nonlinear Signal Extraction . . . . . . . . . . . . . . . . . . . . . . . . . . 50
19.2 The Problem with Nonlinear Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 50
19.3 Monte Carlo Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
19.4 The Smoothed Density . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
20 Lemma III 53
20.1 Intuition and Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
20.2 Proof of Lemma III . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
21 Simulation Smoothing 56
21.1 Applying Lemma III to the State Space Model . . . . . . . . . . . . . . . . . . . . . 57
21.2 Notation Convention . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
21.3 The Unconditional Simulation Step . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
21.4 The Full Simulation Smoothing Algorithm . . . . . . . . . . . . . . . . . . . . . . . . 57
22 The Nonlinear Gaussian Model (nLGM) 58
22.1 Forms of Nonlinearity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
22.2 Example: UK Visits Abroad . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
23 Taylor Expansion and Linearization 59
23.1 Formal Taylor Expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
23.2 The Linearized Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
24 Extended Kalman Filter and Smoother 61
24.1 Parameter Estimation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
25 The Stochastic Volatility Model and QML 63
25.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
25.2 Can We Use the EKF for the SV-Model? . . . . . . . . . . . . . . . . . . . . . . . . 64
25.3 Data Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
25.4 Quasi Maximum Likelihood (QML) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
25.5 The Approximating LGM for QML . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
IV Week 4: Nonlinear Non-Gaussian Models & Mode Estimation 66
26 The Big Picture: From Gaussian to Non-Gaussian 66
27 The Nonlinear Non-Gaussian Model 67
27.1 Model Specification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
27.2 Running Example: Boat Race Wins (Bernoulli) . . . . . . . . . . . . . . . . . . . . . 68
27.3 How the Pieces Fit Together . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
27.4 Why Is the State Equation Still Gaussian? . . . . . . . . . . . . . . . . . . . . . . . . 70
27.5 The Three Model Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
28 Example Models: The Exponential Family 71
28.1 The Exponential Family of Distributions . . . . . . . . . . . . . . . . . . . . . . . . . 71
28.2 Example 1: Poisson Model (Count Data) . . . . . . . . . . . . . . . . . . . . . . . . 72
28.3 Example 2: Binary/Bernoulli Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
28.4 Example 3: Three Special Cases of nLnGMs . . . . . . . . . . . . . . . . . . . . . . . 74
, TSM 2025-2026 Full Study Guide 4
29 Stacked Form Notation 74
29.1 Starting Point: The Per-Period Model . . . . . . . . . . . . . . . . . . . . . . . . . . 74
29.2 Step 1: Stack the Observations and Signals . . . . . . . . . . . . . . . . . . . . . . . 75
29.3 Step 2: The Signal Equation θ = d + Zα . . . . . . . . . . . . . . . . . . . . . . . . . 75
29.4 Step 3: The State Distribution α ∼ N (c, Γ) . . . . . . . . . . . . . . . . . . . . . . . 75
29.5 Step 4: The Observation Equation and the Complete Stacked Model . . . . . . . . . 76
29.6 Key Densities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
29.7 LGM Properties in Stacked Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
30 The Inference Pipeline 78
31 Mode Estimation 86
31.1 The Goal: Why Target the Mode? . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
31.2 The LGM Mode: A Closed-Form Template . . . . . . . . . . . . . . . . . . . . . . . 87
31.3 Newton-Raphson Meets Bayes’ Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
31.4 The Surrogate LGM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
31.5 The Mode Estimation Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
31.6 Two Outputs: The Mode and the Gaussian Approximation . . . . . . . . . . . . . . 97
32 Importance Sampling 98
32.1 Why Importance Sampling? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98
32.2 Signal Extraction: From Weighted Samples to Posterior Quantities . . . . . . . . . . 101
32.3 Prediction: Assembling the Time-Domain Output . . . . . . . . . . . . . . . . . . . . 102
32.4 The SPDK Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
32.5 The Four Densities: Known vs Unknown . . . . . . . . . . . . . . . . . . . . . . . . . 103
33 Signal Extraction Methods: The Full Picture 105
33.1 The Eight Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
33.2 Decision Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
33.3 When to Use What: Summary Table . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
V Week 5: Importance Sampling, Parameter Estimation & Bootstrap Filter107
34 Week 4 Recap: Where We Left Off 107
34.1 The nLnGM Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
34.2 The Five-Step Inference Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
34.3 W4 to W5 Bridge: What We Have vs. What We Need . . . . . . . . . . . . . . . . . 108
34.4 What Week 5 Adds and Why It Matters . . . . . . . . . . . . . . . . . . . . . . . . . 108
35 The Big Picture: Completing the Inference Toolbox 108
35.1 The Signal Extraction Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
36 The Importance Sampling Problem 109
36.1 What We Want . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
36.2 Why Direct Computation Fails . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
37 The SPDK Importance Density 110
37.1 Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
37.2 The Six Densities and Their Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
