Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
LECTURE 1: INTRODUCTION
1.0 General Introduction
Traditionally, problems in statistical inference are divided into problems of estimation
and tests of hypotheses, though actually they are all decision problems. The main
difference between the two kinds of problems is that in problems of estimation we must
determine the value of a parameter (or values of several parameters) from a possible
continuum of alternatives, whereas in tests of hypotheses we must decide whether to
accept or reject a specific value or a set of specific values of a parameter (or those of
several parameters).
This manuscript discusses the various types of point and interval estimators, methods of
determining these estimators, their properties and practical applications.
1.1 Objectives
The overall objective of this manuscript is to outline the methods used in determining
point and interval estimation.
The overall objective will be achieved by considering the following specific objectives:
i. To discuss methods used to determining unbiased, consistent, best asymptotically
normal, sufficient, maximum likelihood, method of moments, and uniformly
minimum variance unbiased estimators of the unknown parameters for probability
distributions.
ii. To examine the fundamental theorems (eg Rao – Blackwell, factorization
criterion, Cramer – Rao lower bound, etc) necessary to undertake the idea of point
estimation.
iii. To examine the Bayesian estimators for unknown parameters for probability
distributions and taking into account the prior information/distribution.
iv. To construct interval estimation for unknown population parameters by assuming
the underlying distributions.
v. To attempt to provide some practical applications of statistical inference as a
subject.
(1)
, Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
1.2 Population Parameters and Sample Statistics
When we use the value of a statistic to estimate a population parameter, we call this point
estimation, and we refer to the value of the statistic as a point estimate of the parameter.
For example, if we use the sample mean, X , to estimate the mean of a population µ , we
are using a point estimate of the population parameter.
The point estimate does not tell us on how much the information is based, nor does it tell
us anything about the possible size of the error. Thus, we might have to supplement a
point estimate θ of θ with the size of the sample and the value of var (θ ) or with some
information about the sampling distribution of θ . An interval estimate of θ is an
interval of the form θ1 < θ < θ 2 , where θ1 and θ 2 are appropriate random values such that
( )
P θ1 < θ < θ 2 = 1 − α
for some specified probability 1 − α . For a specified value of 1 − α , we refer to
θ1 < θ < θ 2 as a (1 − α )100% confidence interval for θ . Also, 1 − α is called the degree
of confidence, and the endpoints of the interval, θ1 and θ 2 , are called the lower and upper
confidence limits.
Since estimators are random variables, one of the key problems of point estimation is to
study their sampling distributions. Various statistical properties of estimators can thus be
used to decide which estimator is most appropriate in a given situation, which will expose
us to the smallest risk, which will give us the most information at the lowest cost, and so
forth. The particular properties that we shall discuss are unbiasedness, minimum
variance, efficiency, consistency, sufficiency, and robustness.
1.3 Summary of Activities
The manuscript is arranged into seven lectures and each of these lectures describes
particular topics of point and interval estimation, present self test exercises and solutions.
Lecture 1 deals with the overall introduction and objectives of the subject area. Lecture 2
describes the properties of unbiasedness, consistency and best asymptotically normal
estimators. In lecture 3, sufficient estimators and factorization criterion are explained. In
addition, the Rao-Blackwell theorem and the property of completeness are described.
Lecture 4 deals with the methods of maximum likelihood estimation and the method of
moments. In lecture 5, efficiency of estimators is considered. This lecture also defines the
Cramer-Rao lower bound theorem and outlines the uniformly minimum variance
unbiased estimators. Lecture 6 describes the method of Bayesian estimation and lectures
7 outlines the interval estimation of the unknown population parameters and presents
some practical applications.
(2)
, Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
LECTURE 2: PROPERTIES OF POINT ESTIMATORS
2.0 Introduction
We shall introduce some of the more common ways by which estimators are categorized.
Some of these are used simply to classify estimators, while others serve as criteria for
selecting a ‘best’ estimator. This lectures describes the properties of unbiased, consistent
and best asymptotically normal estimators.
We begin by stating the following definitions:
Definition 1
Given a set of random variables X 1 , X 2 ,..., X n , a statistic is a function of those random
variables that does not use any unknown parameter values.
Definition 2
Let X 1 , X 2 ,..., X n be i.i.d. random variables with a p.d.f. f X ( x | θ1 , θ 2 ,...,θ k ) and the
statistics gi ( X 1 , X 2 ,..., X n ), i = 1, 2,..., k is determined to be used to estimate the values
of the parameters. Then g1 would be used to estimate the value of θ1 , g 2 to estimate the
value of θ 2 , and so on. These statistics are called estimators of the parameters, and the
values computed for these statistics using sample data are called estimates of the
parameters.
2.1 Unbiased Estimators
One of the most common properties by which estimators are classified is that of
unbiasedness. The idea is that if the average value of a large number of samples is equal
to the true value parameter and is not consistently higher or lower, an estimator then is
classified as being unbiased. A formal definition of this concept follows.
Definition 1
An estimate θ for a parameter θ is said to be unbiased if
E[θ ] = θ .
(3)
, Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
If not, θ is said to be a biased estimator for θ . Furthermore, the quantity
B (θ , θ ) = E[θ ] − θ
is called the bias of θ .
That is, if an estimator for a parameter has a sampling distribution whose mean is equal
to the parameter being estimated, then we have an unbiased estimator and the bias is zero.
If the mean is anything else, then it is a biased estimator and the bias will not be zero. If
the bias is negative, then θ consistently underestimates the actual value of θ . If the bias
is positive, then θ will consistently overestimate the value of θ .
Example 2.1
If X has the binomial distribution with the parameters n and θ , show that the sample
X
proportion, p = ’ is an unbiased estimator of θ .
n
Solution
Since E[ X ] = nθ , it follows that
X 1 1
E[ p] = E = E[ X ] = .nθ = θ
n n n
X
and hence p = is an unbiased estimator of θ .
n
Example 2.2
If X 1 , X 2 ,..., X n constitute a random sample from the population given by
e − ( x − δ ) , x > δ
f ( x) =
0 , elsewhere
show that X is a biased estimator of δ .
(4)
Expanded Features
PDF Unlimited Pages
Documents
Complete
LECTURE 1: INTRODUCTION
1.0 General Introduction
Traditionally, problems in statistical inference are divided into problems of estimation
and tests of hypotheses, though actually they are all decision problems. The main
difference between the two kinds of problems is that in problems of estimation we must
determine the value of a parameter (or values of several parameters) from a possible
continuum of alternatives, whereas in tests of hypotheses we must decide whether to
accept or reject a specific value or a set of specific values of a parameter (or those of
several parameters).
This manuscript discusses the various types of point and interval estimators, methods of
determining these estimators, their properties and practical applications.
1.1 Objectives
The overall objective of this manuscript is to outline the methods used in determining
point and interval estimation.
The overall objective will be achieved by considering the following specific objectives:
i. To discuss methods used to determining unbiased, consistent, best asymptotically
normal, sufficient, maximum likelihood, method of moments, and uniformly
minimum variance unbiased estimators of the unknown parameters for probability
distributions.
ii. To examine the fundamental theorems (eg Rao – Blackwell, factorization
criterion, Cramer – Rao lower bound, etc) necessary to undertake the idea of point
estimation.
iii. To examine the Bayesian estimators for unknown parameters for probability
distributions and taking into account the prior information/distribution.
iv. To construct interval estimation for unknown population parameters by assuming
the underlying distributions.
v. To attempt to provide some practical applications of statistical inference as a
subject.
(1)
, Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
1.2 Population Parameters and Sample Statistics
When we use the value of a statistic to estimate a population parameter, we call this point
estimation, and we refer to the value of the statistic as a point estimate of the parameter.
For example, if we use the sample mean, X , to estimate the mean of a population µ , we
are using a point estimate of the population parameter.
The point estimate does not tell us on how much the information is based, nor does it tell
us anything about the possible size of the error. Thus, we might have to supplement a
point estimate θ of θ with the size of the sample and the value of var (θ ) or with some
information about the sampling distribution of θ . An interval estimate of θ is an
interval of the form θ1 < θ < θ 2 , where θ1 and θ 2 are appropriate random values such that
( )
P θ1 < θ < θ 2 = 1 − α
for some specified probability 1 − α . For a specified value of 1 − α , we refer to
θ1 < θ < θ 2 as a (1 − α )100% confidence interval for θ . Also, 1 − α is called the degree
of confidence, and the endpoints of the interval, θ1 and θ 2 , are called the lower and upper
confidence limits.
Since estimators are random variables, one of the key problems of point estimation is to
study their sampling distributions. Various statistical properties of estimators can thus be
used to decide which estimator is most appropriate in a given situation, which will expose
us to the smallest risk, which will give us the most information at the lowest cost, and so
forth. The particular properties that we shall discuss are unbiasedness, minimum
variance, efficiency, consistency, sufficiency, and robustness.
1.3 Summary of Activities
The manuscript is arranged into seven lectures and each of these lectures describes
particular topics of point and interval estimation, present self test exercises and solutions.
Lecture 1 deals with the overall introduction and objectives of the subject area. Lecture 2
describes the properties of unbiasedness, consistency and best asymptotically normal
estimators. In lecture 3, sufficient estimators and factorization criterion are explained. In
addition, the Rao-Blackwell theorem and the property of completeness are described.
Lecture 4 deals with the methods of maximum likelihood estimation and the method of
moments. In lecture 5, efficiency of estimators is considered. This lecture also defines the
Cramer-Rao lower bound theorem and outlines the uniformly minimum variance
unbiased estimators. Lecture 6 describes the method of Bayesian estimation and lectures
7 outlines the interval estimation of the unknown population parameters and presents
some practical applications.
(2)
, Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
LECTURE 2: PROPERTIES OF POINT ESTIMATORS
2.0 Introduction
We shall introduce some of the more common ways by which estimators are categorized.
Some of these are used simply to classify estimators, while others serve as criteria for
selecting a ‘best’ estimator. This lectures describes the properties of unbiased, consistent
and best asymptotically normal estimators.
We begin by stating the following definitions:
Definition 1
Given a set of random variables X 1 , X 2 ,..., X n , a statistic is a function of those random
variables that does not use any unknown parameter values.
Definition 2
Let X 1 , X 2 ,..., X n be i.i.d. random variables with a p.d.f. f X ( x | θ1 , θ 2 ,...,θ k ) and the
statistics gi ( X 1 , X 2 ,..., X n ), i = 1, 2,..., k is determined to be used to estimate the values
of the parameters. Then g1 would be used to estimate the value of θ1 , g 2 to estimate the
value of θ 2 , and so on. These statistics are called estimators of the parameters, and the
values computed for these statistics using sample data are called estimates of the
parameters.
2.1 Unbiased Estimators
One of the most common properties by which estimators are classified is that of
unbiasedness. The idea is that if the average value of a large number of samples is equal
to the true value parameter and is not consistently higher or lower, an estimator then is
classified as being unbiased. A formal definition of this concept follows.
Definition 1
An estimate θ for a parameter θ is said to be unbiased if
E[θ ] = θ .
(3)
, Click Here & Upgrade
Expanded Features
PDF Unlimited Pages
Documents
Complete
If not, θ is said to be a biased estimator for θ . Furthermore, the quantity
B (θ , θ ) = E[θ ] − θ
is called the bias of θ .
That is, if an estimator for a parameter has a sampling distribution whose mean is equal
to the parameter being estimated, then we have an unbiased estimator and the bias is zero.
If the mean is anything else, then it is a biased estimator and the bias will not be zero. If
the bias is negative, then θ consistently underestimates the actual value of θ . If the bias
is positive, then θ will consistently overestimate the value of θ .
Example 2.1
If X has the binomial distribution with the parameters n and θ , show that the sample
X
proportion, p = ’ is an unbiased estimator of θ .
n
Solution
Since E[ X ] = nθ , it follows that
X 1 1
E[ p] = E = E[ X ] = .nθ = θ
n n n
X
and hence p = is an unbiased estimator of θ .
n
Example 2.2
If X 1 , X 2 ,..., X n constitute a random sample from the population given by
e − ( x − δ ) , x > δ
f ( x) =
0 , elsewhere
show that X is a biased estimator of δ .
(4)
