Summary Chapter14-Econometrics-Stein-ruleEstimation

Chapter 14
Stein-Rule Estimation


The ordinary least squares estimation of regression coefficients in linear regression model provides the
estimators having minimum variance in the class of linear and unbiased estimators. The criterion of
linearity is desirable because such estimators involve less mathematical complexity, they are easy to
compute, and it is easier to investigate their statistical properties. The criterion of unbiasedness is
attractive because it is intuitively desirable to have an estimator whose expected value, i.e., the mean of
the estimator should be the same as the parameter being estimated. Considerations of linearity and
unbiased estimators sometimes may lead to an unacceptably high price to be paid in terms of the
variability around the true parameter. It is possible to have a nonlinear estimator with better properties. It
is to be noted that one of the main objectives of estimation is to find an estimator whose values have high
concentration around the true parameter. Sometimes it is possible to have a nonlinear and biased estimator
that has smaller variability than the variability of a best linear unbiased estimator of the parameter under
some mild restrictions.


In the multiple regression model
y  X    , E     0, V      2 I ,
n1 nk k 1 n1


the ordinary least squares estimator (OLSE) of  is b   X ' X  X ' y which is the best linear unbiased
1



estimator of  in the sense that it is linear in y, E  b    and b has smallest variance among all linear

and unbiased estimators of  . Its covariance matrix is

V  b   E  b    b    '   2  X ' X  .
1




The weighted mean squared error of an estimator ̂ is defined as

   
E ˆ   'W ˆ     wij E ˆi   i
i j
  ˆ j j 
where W is k  k fixed positive definite matrix of weights wij . The two popular choices of weight matrix

W are

(i)   
W is an identity matrix, i.e. W  I then E ˆ   ' ˆ   is called as the total mean squared

error (MSE) of ̂ .


Econometrics | Chapter 14 | Stein-Rule Estimation | Shalabh, IIT Kanpur
1

, (ii) W  X ' X , then

     
E ˆ   ' X ' X ˆ    E X ˆ  X  ' X ˆ  X  

is called as the predictive mean squared error of ̂ . Note that E  y   X ˆ is the predictor of
 
average value E  y   X  and X ˆ  X  is the corresponding prediction error.


There can be other choices of W and it depends entirely on the analyst how to define the loss function so
that the variability is minimum.


If a random vector with k elements  k  2  is normally distributed as N   , I  ,  being the mean vector,

then Stein established that if the linearity and unbiasedness are dropped, then it is possible to improve
upon the maximum likelihood estimator of  under the criterion of total MSE. Later, this result was
generalized by James and Stein for linear regression model. They demonstrated that if the criteria of
linearity and unbiasedness of the estimators are dropped, then a nonlinear estimator can be obtained which
has better performance than the best linear unbiased estimator under the criterion of predictive MSE. In
other words, James and Stein established that OLSE is inadmissible for k  2 under predictive MSE

criterion, i.e., for k  2, there exists an estimator ̂ such that

   
E ˆ   ' X ' X ˆ    E  b    ' X ' X  b   

for all values of  with strict inequality holding for some values of  . For k  2, no such estimator
exists and we say that " b can be beaten” in this sense. Thus it is possible to find estimators which will
beat b in this sense. So a nonlinear and biased estimator can be defined which has better performance
than OLSE. Such an estimator is Stein-rule estimator given by
 2 
ˆ  1  c  b when  2 is known
 b ' X ' Xb 

and
 e 'e 
ˆ  1  c b when  2 is unknown.
 b ' X ' Xb 
Here c is a fixed positive characterizing scalar, e ' e is the residuum sum of squares based on OLSE and
e  y  Xb is the residual. By assuming different values to c , we can generate different estimators. So a
class of estimators characterized by c can be defined. This is called as a family of Stein-rule estimators.

Econometrics | Chapter 14 | Stein-Rule Estimation | Shalabh, IIT Kanpur
2