Chapter 13
Asymptotic Theory and Stochastic Regressors
The nature of explanatory variable is assumed to be non-stochastic or fixed in repeated samples in any
regression analysis. Such an assumption is appropriate for those experiments which are conducted inside the
laboratories where the experimenter can control the values of explanatory variables. Then the repeated
observations on study variable can be obtained for fixed values of explanatory variables. In practice, such
an assumption may not always be satisfied. Sometimes, the explanatory variables in a given model are the
study variable in another model. Thus the study variable depends on the explanatory variables that are
stochastic in nature. Under such situations, the statistical inferences drawn from the linear regression model
based on the assumption of fixed explanatory variables may not remain valid.
We assume now that the explanatory variables are stochastic but uncorrelated with the disturbance term. In
case, they are correlated then the issue is addressed through instrumental variable estimation. Such a
situation arises in the case of measurement error models.
Stochastic regressors model
Consider the linear regression model
y Xβ +ε
=
where X is a (n× k ) matrix of n observations on k explanatory variables X 1 , X 2 ,..., X k which are
stochastic in nature, y is a ( n ×1) vector of n observations on study variable, β is a ( k ×1) vector of
regression coefficients and ε is the ( n ×1) vector of disturbances. Under the assumption
V ( ε ) σ 2 I , the distribution of ε i , conditional on xi' , satisfy these properties for all all values of
E (ε ) 0,=
=
X where xi' denotes the i th row of X . This is demonstrated as follows:
Let p ( ε i | xi' ) be the conditional probability density function of ε i given xi' and p ( ε i ) is the unconditional
probability density function of ε i . Then
E ( ε i | xi' ) = ∫ ε i p ( ε i | xi' ) d ε i
= ∫ ε i p (ε i ) dε i
= E (ε i )
=0
Econometrics | Chapter 13 | Asymptotic Theory and Stochastic Regressors | Shalabh, IIT Kanpur
1
, E ( ε i2 | xi' ) = ∫ ε i2 p ( ε i | xi' ) d ε i
= ∫ ε i2 p ( ε i ) d ε i
= E ( ε i2 )
= σ 2.
In case, ε i and xi' are independent, then p ( ε i | xi' ) = p ( ε i ) .
Least squares estimation of parameters
The additional assumption that the explanatory variables are stochastic poses no problem in the ordinary
least squares estimation of β and σ 2 . The OLSE of β is obtained by minimizing ( y − X β ) ' ( y − X β )
with respect β as
b =(X 'X ) X 'y
−1
and estimator of σ 2 is obtained as
1
s2 = ( y − Xb ) ' ( y − Xb ) .
n−k
Maximum likelihood estimation of parameters:
Assuming ε ~ N ( 0, σ 2 I ) in the model=
y X β + ε along with X is stochastic and independent of ε , the
joint probability density function ε and X can be derived from the joint probability density function of y
and X as follows:
f ( ε , X ) = f ( ε1 , ε 2 ,..., ε n , x1' , x2' ,..., xn' )
n n
= ∏ f ( ε i ) ∏ f ( xi' )
= i 1= i 1
n n
= ∏ f ( yi | xi' ) ∏ f ( xi' )
= i 1= i 1
(
= ∏ f ( yi | xi' ) f ( xi' ) )
n
i =1
= ∏ f ( yi , xi' )
n
i =1
= f ( y1 , y2 ,..., yn , x1' , x2' ,..., xn' )
= f ( y, X ) .
Econometrics | Chapter 13 | Asymptotic Theory and Stochastic Regressors | Shalabh, IIT Kanpur
2
Asymptotic Theory and Stochastic Regressors
The nature of explanatory variable is assumed to be non-stochastic or fixed in repeated samples in any
regression analysis. Such an assumption is appropriate for those experiments which are conducted inside the
laboratories where the experimenter can control the values of explanatory variables. Then the repeated
observations on study variable can be obtained for fixed values of explanatory variables. In practice, such
an assumption may not always be satisfied. Sometimes, the explanatory variables in a given model are the
study variable in another model. Thus the study variable depends on the explanatory variables that are
stochastic in nature. Under such situations, the statistical inferences drawn from the linear regression model
based on the assumption of fixed explanatory variables may not remain valid.
We assume now that the explanatory variables are stochastic but uncorrelated with the disturbance term. In
case, they are correlated then the issue is addressed through instrumental variable estimation. Such a
situation arises in the case of measurement error models.
Stochastic regressors model
Consider the linear regression model
y Xβ +ε
=
where X is a (n× k ) matrix of n observations on k explanatory variables X 1 , X 2 ,..., X k which are
stochastic in nature, y is a ( n ×1) vector of n observations on study variable, β is a ( k ×1) vector of
regression coefficients and ε is the ( n ×1) vector of disturbances. Under the assumption
V ( ε ) σ 2 I , the distribution of ε i , conditional on xi' , satisfy these properties for all all values of
E (ε ) 0,=
=
X where xi' denotes the i th row of X . This is demonstrated as follows:
Let p ( ε i | xi' ) be the conditional probability density function of ε i given xi' and p ( ε i ) is the unconditional
probability density function of ε i . Then
E ( ε i | xi' ) = ∫ ε i p ( ε i | xi' ) d ε i
= ∫ ε i p (ε i ) dε i
= E (ε i )
=0
Econometrics | Chapter 13 | Asymptotic Theory and Stochastic Regressors | Shalabh, IIT Kanpur
1
, E ( ε i2 | xi' ) = ∫ ε i2 p ( ε i | xi' ) d ε i
= ∫ ε i2 p ( ε i ) d ε i
= E ( ε i2 )
= σ 2.
In case, ε i and xi' are independent, then p ( ε i | xi' ) = p ( ε i ) .
Least squares estimation of parameters
The additional assumption that the explanatory variables are stochastic poses no problem in the ordinary
least squares estimation of β and σ 2 . The OLSE of β is obtained by minimizing ( y − X β ) ' ( y − X β )
with respect β as
b =(X 'X ) X 'y
−1
and estimator of σ 2 is obtained as
1
s2 = ( y − Xb ) ' ( y − Xb ) .
n−k
Maximum likelihood estimation of parameters:
Assuming ε ~ N ( 0, σ 2 I ) in the model=
y X β + ε along with X is stochastic and independent of ε , the
joint probability density function ε and X can be derived from the joint probability density function of y
and X as follows:
f ( ε , X ) = f ( ε1 , ε 2 ,..., ε n , x1' , x2' ,..., xn' )
n n
= ∏ f ( ε i ) ∏ f ( xi' )
= i 1= i 1
n n
= ∏ f ( yi | xi' ) ∏ f ( xi' )
= i 1= i 1
(
= ∏ f ( yi | xi' ) f ( xi' ) )
n
i =1
= ∏ f ( yi , xi' )
n
i =1
= f ( y1 , y2 ,..., yn , x1' , x2' ,..., xn' )
= f ( y, X ) .
Econometrics | Chapter 13 | Asymptotic Theory and Stochastic Regressors | Shalabh, IIT Kanpur
2
