Econometrics
Page 1 of 49
,C ONTENTS
1 Lecture one: A statistics refresher.......................................................................................... 6
1.1 What is econometrics?.................................................................................................... 6
1.2 Univariate analysis (summarizing one variable) ................................................................ 6
1.2.1 Summarizing the variable in words ................................................................................................................. 6
1.2.2 Summarizing the variable in numbers ............................................................................................................ 7
1.2.3 Expected value and its rules of calculation ..................................................................................................... 7
1.2.4 The variance .................................................................................................................................................... 8
1.2.5 The standard deviation ................................................................................................................................... 8
1.2.6 The variance calculation rules ......................................................................................................................... 8
1.2.7 Covariance ....................................................................................................................................................... 9
1.3 Bivariate analysis ........................................................................................................... 9
1.3.1 Types of distributions ...................................................................................................................................... 9
1.3.2 Using distributions........................................................................................................................................... 9
1.3.3 Conditional distributions ................................................................................................................................. 9
1.3.4 Applying the conditional distribution............................................................................................................ 10
1.4 Independence................................................................................................................10
1.4.1 Conditional expectations............................................................................................................................... 11
1.4.2 Conditional expectations calculation rules ................................................................................................... 11
1.5 Covariance....................................................................................................................11
1.5.1 Covariance calculation rules.......................................................................................................................... 11
1.6 Correlation....................................................................................................................12
1.6.1 Rules of calculation ....................................................................................................................................... 12
1.7 The sample ...................................................................................................................12
1.7.1 The sample mean .......................................................................................................................................... 12
1.7.2 The sample variance...................................................................................................................................... 12
1.7.3 The sample covariance.................................................................................................................................. 13
1.7.4 The sample correlation.................................................................................................................................. 13
1.8 The terminology of the regression model ........................................................................13
2 Lecture 2: A closer look at regressions ..................................................................................13
2.1 Bivariate regression analysis ..........................................................................................13
2.1.1 The regression ............................................................................................................................................... 13
2.1.2 Ordinary Least Squares (OLS) ........................................................................................................................ 14
2.1.3 R-squared ...................................................................................................................................................... 14
2.1.4 Root mean squared error (Root MSE)........................................................................................................... 15
2.1.5 Multivariate regression ................................................................................................................................. 15
2.1.6 What the residuals tell us.............................................................................................................................. 15
2.2 The assumptions of the OLS estimator ............................................................................15
2.2.1 Assumption 1: Population model is linear in parameters............................................................................. 16
Page 2 of 49
, 2.2.2 Assumption 2: Error term has a zero population mean................................................................................ 16
2.2.3 Assumption 3: All independent variables are uncorrelated with the error term......................................... 16
2.2.4 Assumption 4: No perfect (multi)collinearity between independent variables ........................................... 16
2.2.5 Assumption 5: No serial correlation.............................................................................................................. 16
2.2.6 Assumption 6: No heteroskedasticity ........................................................................................................... 16
2.3 The variance of the estimator.........................................................................................17
2.3.1 The intuition of the variance of the OLS estimator....................................................................................... 17
2.3.2 We however do not observe 𝝈𝟐 at all.......................................................................................................... 17
2.3.3 From STATA output to the estimated variance of the OLS estimator .......................................................... 18
3 Lecture 3: hypothesis testing and the omitted variable bias...................................................18
3.1.1 Assumption 7: Normality of the error term .................................................................................................. 18
3.2 Hypothesis testing .........................................................................................................19
3.2.1 Stating the hypotheses.................................................................................................................................. 19
3.2.2 Testing the hypothesis .................................................................................................................................. 19
3.2.3 The T-statistic ................................................................................................................................................ 20
3.2.4 The p-value .................................................................................................................................................... 20
3.2.5 The confidence interval ................................................................................................................................. 21
3.3 Joint hypothesis testing: the F-test..................................................................................21
3.3.1 Conducting the F-test.................................................................................................................................... 21
3.4 Omitted Variable Bias ....................................................................................................23
4 Lecture 4: The functional regression equation.......................................................................23
4.1 Rescaling a variable.......................................................................................................23
4.2 Alternative functional forms...........................................................................................24
4.2.1 Linear form .................................................................................................................................................... 24
4.2.2 Double-log Form............................................................................................................................................ 24
4.2.3 Semilog form ................................................................................................................................................. 24
4.2.4 Polynomial form ............................................................................................................................................ 25
4.2.5 The marginal effect of a quadratic term ....................................................................................................... 25
4.3 Slope dummy variables ..................................................................................................26
4.4 Testing the differences between dummy groups – the Chow test......................................26
4.4.1 What is the chow test and what does it mean?............................................................................................ 27
4.4.2 How does the Chow test work? .................................................................................................................... 27
4.5 Other dummy variables..................................................................................................28
4.5.1 What if you have dummies with more than 2 groups? ................................................................................ 28
4.6 Problems with incorrect functional forms ........................................................................29
4.6.1 Estimated R-squared is difficult to compare if the IV is transformed........................................................... 29
4.6.2 Incorrect functional forms outside the range of the sample........................................................................ 29
5 Lecture 5: Multicollinearity and Heteroskedasticity ...............................................................29
5.1 Multicollinearity ............................................................................................................29
Page 3 of 49
, 5.1.1 Perfect multicollinearity................................................................................................................................ 30
5.1.2 Imperfect multicollinearity............................................................................................................................ 30
5.1.3 Solutions to multicollinearity ........................................................................................................................ 30
5.1.4 How to describe this problem ....................................................................................................................... 30
5.2 Heteroskedasticity .........................................................................................................31
5.2.1 Testing for heteroskedasticity....................................................................................................................... 31
5.2.2 The solution or cure for heteroskedasticity .................................................................................................. 32
5.2.3 Disease-consequence-diagnosis-cure ........................................................................................................... 32
6 Lecture 6: time-series models ...............................................................................................32
6.1 What is a time series model?..........................................................................................32
6.1.1 Three types of models................................................................................................................................... 33
6.1.2 Static time-series model................................................................................................................................ 33
6.1.3 Distributed lag model .................................................................................................................................... 33
6.1.4 Autoregressive distributed lag model ........................................................................................................... 34
6.2 Issues with time series models ........................................................................................34
6.2.1 Spurious regression ....................................................................................................................................... 34
6.2.2 Seasonality..................................................................................................................................................... 34
6.2.3 Serial correlation ........................................................................................................................................... 35
6.2.4 How to test for serial correlation: The Breusch-Godfrey test....................................................................... 36
6.2.5 How to correct for serial correlation............................................................................................................. 36
7 Lecture 7: Different time series models .................................................................................37
7.1 Dynamic models............................................................................................................37
7.2 Exogeneity ....................................................................................................................38
7.3 Stationarity...................................................................................................................38
7.3.1 What is stationarity? ..................................................................................................................................... 38
7.4 Examples of stationary time series..................................................................................39
7.4.1 White noise ................................................................................................................................................... 39
7.4.2 Stable AR(1) time-series ................................................................................................................................ 39
7.5 Examples of non-stationary time series...........................................................................42
7.5.1 Trending variables ......................................................................................................................................... 42
7.5.2 Random walk................................................................................................................................................. 42
7.6 Diagnosis of non-stationarity .........................................................................................44
7.6.1 -Understanding the dickey-fuller test ........................................................................................................... 44
7.6.2 Performing the Dickey-Fuller test for unit root ............................................................................................ 44
7.6.3 Informal evidence.......................................................................................................................................... 44
7.7 When to use first-difference and when not......................................................................44
7.7.1 Cointegration................................................................................................................................................. 45
8 Lecture 8: Linear probability model.......................................................................................45
8.1 Revisiting dummy variables ............................................................................................45
Page 4 of 49
, 8.1.1 The dummy as an independent variable ....................................................................................................... 45
8.1.2 The dummy as the dependent variable ........................................................................................................ 45
8.2 Disadvantages of using dummy variables........................................................................47
8.2.1 Heteroskedastic errors .................................................................................................................................. 47
8.2.2 Predicted probabilities outside the 0-1 interval ........................................................................................... 47
9 The cheat sheet....................................................................................................................47
9.1 Some final notes on causality .........................................................................................47
9.2 Calculation rules............................................................................................................48
9.3 Formulas.......................................................................................................................48
9.4 The sample and the population ......................................................................................48
9.5 How to: systematic hypothesis testing ............................................................................49
9.6 Random notes...............................................................................................................49
Page 5 of 49
,1 L ECTURE ONE : A STATISTICS REFRESHER
1.1 W HAT IS ECONOMETRICS?
We study econometrics to measure causal effects. Mostly, these are supported or indicated by
economic theory. We however need a way to give empirical evidence. We need to test these
relationships (theorized by theory) in the real world.
Ideally, we would want to conduct experiments. In such a test, we would be able to control every
factor, making it the optimal way of measuring something. Imagine however that we would conduct
an experiment on college degree and standards of living. It would be impossible to conduct such a
research with an experimental approach. Thus, we move to observational non-experimental data.
1.2 U NIVARIATE ANALYSIS (SUMMARIZING ONE VARIABLE)
1.2.1 Summarizing the variable in words
At a univariate analysis, we look at one random variable and try to describe it. The thing with a random
variable, is that it can take on a random variable. We can write that as follows:
X = a variable that takes on different values
x i = a random value
So we want to take a look at:
𝑃𝑟(𝑋 = 𝑥 i)
The probability that the (random) variable X takes on the value of x i
What you see above, applies to our sample which only delivers us a handful of those probabilities. Our
population covers all these probabilities. Thus, we can write that as a function of all the probabilities:
𝑓 (𝑥i ) = 𝑃𝑟(𝑋 = 𝑥 i)
This is called the Probability Density Function. It shows the probability for each of the different
outcomes. Some values are more likely to occur than others, which increases their probability. If plot
such probabilities, we achieve a pdf, probability density function.
Page 6 of 49
, There are two different probability density functions: a discrete and continuous. A discrete pdf works
for countable outcomes and a continuous pdf works for non-countable outcomes
A discrete pdf is what we are used to the most. It has a few important properties:
1. When 𝑥 i is positive, the Pr is positive:
𝑃𝑟(𝑋 = 𝑥 i) ≥ 0 𝑓𝑜𝑟 𝑖 = 1,2,3, . . , 𝑁
2. If you sum all the probabilities, you will get to one:
𝑁
∑ 𝑃𝑟 (𝑋 = 𝑥i) = 1
𝑖=1
Or…
𝑁
∑ 𝑓(𝑥 i) = 1
𝑖=1
1.2.2 Summarizing the variable in numbers
In simple terms, the expected value of a random variable is its average. We get to such an average
value by weighing each observed value with the probability that it occurs. Thus:
𝑁
𝐸(𝑋) = 𝐸𝑋 = 𝜇 𝑋 = ∑ 𝑃𝑟(𝑋 = 𝑥 𝑖 ) × 𝑥𝑖
𝑖=1
That last part states: “The sum of 𝑥 𝑖 multiplied by the probability of 𝑋 taking on the value 𝑥 𝑖”
1.2.2.1 Expected value of rolling a dice
Let’s take that and put it to test with rolling a dice. We can create the following table:
𝒙𝐢 𝑷𝒓(𝑿 = 𝒙𝒊 ) 𝑷𝒓(𝑿 = 𝒙𝒊 ) × 𝒙𝒊
1 1/6 1/6
2 1/6 2/6
3 1/6 3/6
4 1/6 4/6
5 1/6 5/6
6 1/6 6/6
( )
𝐸 𝑋 = 21/6 = 3.5
1.2.3 Expected value and its rules of calculation
1. The expected value of a constant, is the constant:
𝐸 (𝑐) = 𝑐
2. The expected value of a constant plus a random variable is the expected value of the random
variable plus real value of the constant:
𝐸 (𝑋 + 𝑐) = 𝐸(𝑋) + 𝑐 = µ𝑋 + 𝑐
3. The expected value of a random variable multiplied by a constant is the constant multiplied by
the expected value of a random variable:
𝐸 (𝑐𝑋) = 𝑐𝐸(𝑋) = 𝑐µ𝑋
4. When two random variables are summed, the expected value of that sum is the summation of
the expected values of both variables apart:
𝐸 (𝑋1 + 𝑋2) = 𝐸(𝑋1) + 𝐸(𝑋2) = µ𝑋1 + µ𝑋2
Page 7 of 49
Page 1 of 49
,C ONTENTS
1 Lecture one: A statistics refresher.......................................................................................... 6
1.1 What is econometrics?.................................................................................................... 6
1.2 Univariate analysis (summarizing one variable) ................................................................ 6
1.2.1 Summarizing the variable in words ................................................................................................................. 6
1.2.2 Summarizing the variable in numbers ............................................................................................................ 7
1.2.3 Expected value and its rules of calculation ..................................................................................................... 7
1.2.4 The variance .................................................................................................................................................... 8
1.2.5 The standard deviation ................................................................................................................................... 8
1.2.6 The variance calculation rules ......................................................................................................................... 8
1.2.7 Covariance ....................................................................................................................................................... 9
1.3 Bivariate analysis ........................................................................................................... 9
1.3.1 Types of distributions ...................................................................................................................................... 9
1.3.2 Using distributions........................................................................................................................................... 9
1.3.3 Conditional distributions ................................................................................................................................. 9
1.3.4 Applying the conditional distribution............................................................................................................ 10
1.4 Independence................................................................................................................10
1.4.1 Conditional expectations............................................................................................................................... 11
1.4.2 Conditional expectations calculation rules ................................................................................................... 11
1.5 Covariance....................................................................................................................11
1.5.1 Covariance calculation rules.......................................................................................................................... 11
1.6 Correlation....................................................................................................................12
1.6.1 Rules of calculation ....................................................................................................................................... 12
1.7 The sample ...................................................................................................................12
1.7.1 The sample mean .......................................................................................................................................... 12
1.7.2 The sample variance...................................................................................................................................... 12
1.7.3 The sample covariance.................................................................................................................................. 13
1.7.4 The sample correlation.................................................................................................................................. 13
1.8 The terminology of the regression model ........................................................................13
2 Lecture 2: A closer look at regressions ..................................................................................13
2.1 Bivariate regression analysis ..........................................................................................13
2.1.1 The regression ............................................................................................................................................... 13
2.1.2 Ordinary Least Squares (OLS) ........................................................................................................................ 14
2.1.3 R-squared ...................................................................................................................................................... 14
2.1.4 Root mean squared error (Root MSE)........................................................................................................... 15
2.1.5 Multivariate regression ................................................................................................................................. 15
2.1.6 What the residuals tell us.............................................................................................................................. 15
2.2 The assumptions of the OLS estimator ............................................................................15
2.2.1 Assumption 1: Population model is linear in parameters............................................................................. 16
Page 2 of 49
, 2.2.2 Assumption 2: Error term has a zero population mean................................................................................ 16
2.2.3 Assumption 3: All independent variables are uncorrelated with the error term......................................... 16
2.2.4 Assumption 4: No perfect (multi)collinearity between independent variables ........................................... 16
2.2.5 Assumption 5: No serial correlation.............................................................................................................. 16
2.2.6 Assumption 6: No heteroskedasticity ........................................................................................................... 16
2.3 The variance of the estimator.........................................................................................17
2.3.1 The intuition of the variance of the OLS estimator....................................................................................... 17
2.3.2 We however do not observe 𝝈𝟐 at all.......................................................................................................... 17
2.3.3 From STATA output to the estimated variance of the OLS estimator .......................................................... 18
3 Lecture 3: hypothesis testing and the omitted variable bias...................................................18
3.1.1 Assumption 7: Normality of the error term .................................................................................................. 18
3.2 Hypothesis testing .........................................................................................................19
3.2.1 Stating the hypotheses.................................................................................................................................. 19
3.2.2 Testing the hypothesis .................................................................................................................................. 19
3.2.3 The T-statistic ................................................................................................................................................ 20
3.2.4 The p-value .................................................................................................................................................... 20
3.2.5 The confidence interval ................................................................................................................................. 21
3.3 Joint hypothesis testing: the F-test..................................................................................21
3.3.1 Conducting the F-test.................................................................................................................................... 21
3.4 Omitted Variable Bias ....................................................................................................23
4 Lecture 4: The functional regression equation.......................................................................23
4.1 Rescaling a variable.......................................................................................................23
4.2 Alternative functional forms...........................................................................................24
4.2.1 Linear form .................................................................................................................................................... 24
4.2.2 Double-log Form............................................................................................................................................ 24
4.2.3 Semilog form ................................................................................................................................................. 24
4.2.4 Polynomial form ............................................................................................................................................ 25
4.2.5 The marginal effect of a quadratic term ....................................................................................................... 25
4.3 Slope dummy variables ..................................................................................................26
4.4 Testing the differences between dummy groups – the Chow test......................................26
4.4.1 What is the chow test and what does it mean?............................................................................................ 27
4.4.2 How does the Chow test work? .................................................................................................................... 27
4.5 Other dummy variables..................................................................................................28
4.5.1 What if you have dummies with more than 2 groups? ................................................................................ 28
4.6 Problems with incorrect functional forms ........................................................................29
4.6.1 Estimated R-squared is difficult to compare if the IV is transformed........................................................... 29
4.6.2 Incorrect functional forms outside the range of the sample........................................................................ 29
5 Lecture 5: Multicollinearity and Heteroskedasticity ...............................................................29
5.1 Multicollinearity ............................................................................................................29
Page 3 of 49
, 5.1.1 Perfect multicollinearity................................................................................................................................ 30
5.1.2 Imperfect multicollinearity............................................................................................................................ 30
5.1.3 Solutions to multicollinearity ........................................................................................................................ 30
5.1.4 How to describe this problem ....................................................................................................................... 30
5.2 Heteroskedasticity .........................................................................................................31
5.2.1 Testing for heteroskedasticity....................................................................................................................... 31
5.2.2 The solution or cure for heteroskedasticity .................................................................................................. 32
5.2.3 Disease-consequence-diagnosis-cure ........................................................................................................... 32
6 Lecture 6: time-series models ...............................................................................................32
6.1 What is a time series model?..........................................................................................32
6.1.1 Three types of models................................................................................................................................... 33
6.1.2 Static time-series model................................................................................................................................ 33
6.1.3 Distributed lag model .................................................................................................................................... 33
6.1.4 Autoregressive distributed lag model ........................................................................................................... 34
6.2 Issues with time series models ........................................................................................34
6.2.1 Spurious regression ....................................................................................................................................... 34
6.2.2 Seasonality..................................................................................................................................................... 34
6.2.3 Serial correlation ........................................................................................................................................... 35
6.2.4 How to test for serial correlation: The Breusch-Godfrey test....................................................................... 36
6.2.5 How to correct for serial correlation............................................................................................................. 36
7 Lecture 7: Different time series models .................................................................................37
7.1 Dynamic models............................................................................................................37
7.2 Exogeneity ....................................................................................................................38
7.3 Stationarity...................................................................................................................38
7.3.1 What is stationarity? ..................................................................................................................................... 38
7.4 Examples of stationary time series..................................................................................39
7.4.1 White noise ................................................................................................................................................... 39
7.4.2 Stable AR(1) time-series ................................................................................................................................ 39
7.5 Examples of non-stationary time series...........................................................................42
7.5.1 Trending variables ......................................................................................................................................... 42
7.5.2 Random walk................................................................................................................................................. 42
7.6 Diagnosis of non-stationarity .........................................................................................44
7.6.1 -Understanding the dickey-fuller test ........................................................................................................... 44
7.6.2 Performing the Dickey-Fuller test for unit root ............................................................................................ 44
7.6.3 Informal evidence.......................................................................................................................................... 44
7.7 When to use first-difference and when not......................................................................44
7.7.1 Cointegration................................................................................................................................................. 45
8 Lecture 8: Linear probability model.......................................................................................45
8.1 Revisiting dummy variables ............................................................................................45
Page 4 of 49
, 8.1.1 The dummy as an independent variable ....................................................................................................... 45
8.1.2 The dummy as the dependent variable ........................................................................................................ 45
8.2 Disadvantages of using dummy variables........................................................................47
8.2.1 Heteroskedastic errors .................................................................................................................................. 47
8.2.2 Predicted probabilities outside the 0-1 interval ........................................................................................... 47
9 The cheat sheet....................................................................................................................47
9.1 Some final notes on causality .........................................................................................47
9.2 Calculation rules............................................................................................................48
9.3 Formulas.......................................................................................................................48
9.4 The sample and the population ......................................................................................48
9.5 How to: systematic hypothesis testing ............................................................................49
9.6 Random notes...............................................................................................................49
Page 5 of 49
,1 L ECTURE ONE : A STATISTICS REFRESHER
1.1 W HAT IS ECONOMETRICS?
We study econometrics to measure causal effects. Mostly, these are supported or indicated by
economic theory. We however need a way to give empirical evidence. We need to test these
relationships (theorized by theory) in the real world.
Ideally, we would want to conduct experiments. In such a test, we would be able to control every
factor, making it the optimal way of measuring something. Imagine however that we would conduct
an experiment on college degree and standards of living. It would be impossible to conduct such a
research with an experimental approach. Thus, we move to observational non-experimental data.
1.2 U NIVARIATE ANALYSIS (SUMMARIZING ONE VARIABLE)
1.2.1 Summarizing the variable in words
At a univariate analysis, we look at one random variable and try to describe it. The thing with a random
variable, is that it can take on a random variable. We can write that as follows:
X = a variable that takes on different values
x i = a random value
So we want to take a look at:
𝑃𝑟(𝑋 = 𝑥 i)
The probability that the (random) variable X takes on the value of x i
What you see above, applies to our sample which only delivers us a handful of those probabilities. Our
population covers all these probabilities. Thus, we can write that as a function of all the probabilities:
𝑓 (𝑥i ) = 𝑃𝑟(𝑋 = 𝑥 i)
This is called the Probability Density Function. It shows the probability for each of the different
outcomes. Some values are more likely to occur than others, which increases their probability. If plot
such probabilities, we achieve a pdf, probability density function.
Page 6 of 49
, There are two different probability density functions: a discrete and continuous. A discrete pdf works
for countable outcomes and a continuous pdf works for non-countable outcomes
A discrete pdf is what we are used to the most. It has a few important properties:
1. When 𝑥 i is positive, the Pr is positive:
𝑃𝑟(𝑋 = 𝑥 i) ≥ 0 𝑓𝑜𝑟 𝑖 = 1,2,3, . . , 𝑁
2. If you sum all the probabilities, you will get to one:
𝑁
∑ 𝑃𝑟 (𝑋 = 𝑥i) = 1
𝑖=1
Or…
𝑁
∑ 𝑓(𝑥 i) = 1
𝑖=1
1.2.2 Summarizing the variable in numbers
In simple terms, the expected value of a random variable is its average. We get to such an average
value by weighing each observed value with the probability that it occurs. Thus:
𝑁
𝐸(𝑋) = 𝐸𝑋 = 𝜇 𝑋 = ∑ 𝑃𝑟(𝑋 = 𝑥 𝑖 ) × 𝑥𝑖
𝑖=1
That last part states: “The sum of 𝑥 𝑖 multiplied by the probability of 𝑋 taking on the value 𝑥 𝑖”
1.2.2.1 Expected value of rolling a dice
Let’s take that and put it to test with rolling a dice. We can create the following table:
𝒙𝐢 𝑷𝒓(𝑿 = 𝒙𝒊 ) 𝑷𝒓(𝑿 = 𝒙𝒊 ) × 𝒙𝒊
1 1/6 1/6
2 1/6 2/6
3 1/6 3/6
4 1/6 4/6
5 1/6 5/6
6 1/6 6/6
( )
𝐸 𝑋 = 21/6 = 3.5
1.2.3 Expected value and its rules of calculation
1. The expected value of a constant, is the constant:
𝐸 (𝑐) = 𝑐
2. The expected value of a constant plus a random variable is the expected value of the random
variable plus real value of the constant:
𝐸 (𝑋 + 𝑐) = 𝐸(𝑋) + 𝑐 = µ𝑋 + 𝑐
3. The expected value of a random variable multiplied by a constant is the constant multiplied by
the expected value of a random variable:
𝐸 (𝑐𝑋) = 𝑐𝐸(𝑋) = 𝑐µ𝑋
4. When two random variables are summed, the expected value of that sum is the summation of
the expected values of both variables apart:
𝐸 (𝑋1 + 𝑋2) = 𝐸(𝑋1) + 𝐸(𝑋2) = µ𝑋1 + µ𝑋2
Page 7 of 49
