7) Statistical Data Treatment and Evaluation
Introduction
The consequences of making errors in statistical tests are often compared with the consequences of errors
made in disciplinary (judicial) procedures. Consider a student in a boarding secondary facing a school
disciplinary committee, comprising of 12 teachers, over suspected theft.
After hearing the case, one of the 12 teachers does not agree with the others, who are trying to convict her
to expulsion from the school.
In this disciplinary hearing, we can make two types of errors. An innocent student can be expelled or a
guilty student can be set free, and retained in the school.
In the justice system, it is considered a more serious error to convict an innocent person than to acquit a
guilty one.
Similarly, in statistical tests to determine whether two quantities are the same, two types of errors can be
made:
(i) A type I error occurs when we reject the hypothesis that two quantities are the same when they are
statistically identical.
(ii) A type II error occurs when we accept that they are the same when they are not statistically identical.
The characteristics of these errors in statistical testing and the ways we can minimize them are among the
subjects of this chapter.
Scientists use statistical calculations to sharpen their judgments concerning the quality of experimental
measurements. We consider several of the most common applications of statistical tests to the treatment
of analytical results.
These applications in include the following:
1. Defining a numerical interval around the mean of a set of replicate analytical results within which the
population mean can be expected to lie with a certain probability. This interval is called the confidence
interval (CI). The interval is related to the standard deviation of the mean.
2. Determining the number of replicate measurements required to ensure that an experimental mean falls
within a certain range with a given level of probability.
3. Estimating the probability that (a) an experimental mean and a true value or (b) two experimental
means are different; that is, whether the difference is real or simply the result of random error. This
test is particularly important for discovering systematic errors in a method and determining whether
two samples come from the same source.
4. Determining at a given probability level whether the precision of two sets of measurements differs.
5. Comparing the means of more than two samples to determine whether differences in the means are
real or the result of random error. This process is known as analysis of variance.
6. Deciding with a certain probability whether an apparent outlier in a set of replicate measurements is
the result of a gross error and can thus be rejected or whether it is a legitimate part of the population
that must be retained in calculating the mean of the set.
1) Confidence Intervals
In most of the situations encountered in chemical analysis, the true value of the mean μ cannot be
determined because a huge number of measurements (approaching infinity) would be required.
̅
With statistics, however, we can establish an interval surrounding an experimentally determined mean 𝐗
within which the population mean μ is expected to lie with a certain degree of probability. This interval is
known as the confidence interval and the boundaries are called confidence limits.
, For example, we might say that it is 99% probable that the true population mean for a set of potassium
measurements lies in the interval 7.25% ± 0.15% K. Thus, the mean should lie in the interval from 7.10%
to 7.40% K with 99% probability.
The size of the confidence interval which is computed from the sample standard deviation, depends on
how well the sample standard deviation s estimates the population standard deviation 𝛔. If s is a good
approximation of 𝝈, the confidence interval can be significantly narrower than if the estimate of 𝛔 is based
on only a few measurement values.
(a) Finding the confidence interval when 𝛔 is known or s is a good
estimate of 𝛔
Figure 1 shows a series of five normal error curves.
Figure 1
In each, the relative frequency is plotted as a function of the quantity z, which is the deviation from
the mean divided by the population standard deviation.
Equation 1
The shaded areas in each plot lie between the values of -z and +z: that are indicated to the left
and right of the curves. The numbers within the shaded areas are the percentage of the total area
under the curve that is included within these values of z. For example as shown in curve:
(a) 50% of the area under any Gaussian curve is located between -0.67σ and +0.67 σ.
(b) 80% of the total area lies between -1.28 σ and + 1.28 σ
(c) 90% lies between -1.64 σ and + 1.64 σ
(d) 95% lies between -1.96 σ and + 1.96 σ
(e) 99% lies between -2.58 σ and + 2.58 σ
Relationships such as these allow us to define a range of values around a measurement result
within which the true mean is likely to lie with a certain probability provided we have a reasonable
estimate of σ.
, For example, if we have a result x from a data set with a standard deviation of σ, we may assume
that 90 times out of 100, the true mean μ will fall in the interval x ± 1.64σ (see Figure c). The
probability is called the confidence level (CL).
In the example of Figure c, the confidence level is 90%, and the confidence interval is from -
1.64σ to +1.64 σ. The probability that a result is outside the confidence interval is often called
the significance level.
If we make a single measurement x from a distribution of known σ, we can say that the true mean
should lie in the interval x ± 𝑧σ with a probability dependent on z. This probability is 90% for z
= 1.64, 95% for z = 1.96, and 99% for z = 2.58, as shown in Figure c, d, and e.
We find a general expression for the confidence interval of the true mean based on measuring a
single value x by rearranging Equation 1. (Remember that z can take positive or negative values.)
Thus,
Confidence limit for 𝜇 = x ± 𝑧𝜎 Equation 2
Table 1
Rarely do we estimate the true mean from a single measurement, however. Instead, we use the
̅ of N measurements as a better estimate of μ. In this case, we replace x in
experimental mean 𝐗
̅ and σ with the standard error of the mean, σ/√𝑁. That is:
Equation 1 with 𝐗
𝒁𝛔
Confidence limit (CL) of 𝜇 = ̅
X ± Equation 3
√𝑵
Example 1
Glucose levels are routinely monitored in patients suffering from diabetes. The glucose
concentrations in a patient with mildly elevated glucose levels were determined in different months
by a spectrophotometric analytical method. The patient was placed on a low-sugar diet to reduce the
glucose levels. The following results were obtained during a study to determine the effectiveness of
the diet. Calculate a pooled estimate of the standard deviation for the method.
Time Glucose Concentration, mg/L
Month 1 1108, 1122, 1075, 1099, 1115, 1083, 1100
Month 2 992, 975, 1022, 1001, 991
Introduction
The consequences of making errors in statistical tests are often compared with the consequences of errors
made in disciplinary (judicial) procedures. Consider a student in a boarding secondary facing a school
disciplinary committee, comprising of 12 teachers, over suspected theft.
After hearing the case, one of the 12 teachers does not agree with the others, who are trying to convict her
to expulsion from the school.
In this disciplinary hearing, we can make two types of errors. An innocent student can be expelled or a
guilty student can be set free, and retained in the school.
In the justice system, it is considered a more serious error to convict an innocent person than to acquit a
guilty one.
Similarly, in statistical tests to determine whether two quantities are the same, two types of errors can be
made:
(i) A type I error occurs when we reject the hypothesis that two quantities are the same when they are
statistically identical.
(ii) A type II error occurs when we accept that they are the same when they are not statistically identical.
The characteristics of these errors in statistical testing and the ways we can minimize them are among the
subjects of this chapter.
Scientists use statistical calculations to sharpen their judgments concerning the quality of experimental
measurements. We consider several of the most common applications of statistical tests to the treatment
of analytical results.
These applications in include the following:
1. Defining a numerical interval around the mean of a set of replicate analytical results within which the
population mean can be expected to lie with a certain probability. This interval is called the confidence
interval (CI). The interval is related to the standard deviation of the mean.
2. Determining the number of replicate measurements required to ensure that an experimental mean falls
within a certain range with a given level of probability.
3. Estimating the probability that (a) an experimental mean and a true value or (b) two experimental
means are different; that is, whether the difference is real or simply the result of random error. This
test is particularly important for discovering systematic errors in a method and determining whether
two samples come from the same source.
4. Determining at a given probability level whether the precision of two sets of measurements differs.
5. Comparing the means of more than two samples to determine whether differences in the means are
real or the result of random error. This process is known as analysis of variance.
6. Deciding with a certain probability whether an apparent outlier in a set of replicate measurements is
the result of a gross error and can thus be rejected or whether it is a legitimate part of the population
that must be retained in calculating the mean of the set.
1) Confidence Intervals
In most of the situations encountered in chemical analysis, the true value of the mean μ cannot be
determined because a huge number of measurements (approaching infinity) would be required.
̅
With statistics, however, we can establish an interval surrounding an experimentally determined mean 𝐗
within which the population mean μ is expected to lie with a certain degree of probability. This interval is
known as the confidence interval and the boundaries are called confidence limits.
, For example, we might say that it is 99% probable that the true population mean for a set of potassium
measurements lies in the interval 7.25% ± 0.15% K. Thus, the mean should lie in the interval from 7.10%
to 7.40% K with 99% probability.
The size of the confidence interval which is computed from the sample standard deviation, depends on
how well the sample standard deviation s estimates the population standard deviation 𝛔. If s is a good
approximation of 𝝈, the confidence interval can be significantly narrower than if the estimate of 𝛔 is based
on only a few measurement values.
(a) Finding the confidence interval when 𝛔 is known or s is a good
estimate of 𝛔
Figure 1 shows a series of five normal error curves.
Figure 1
In each, the relative frequency is plotted as a function of the quantity z, which is the deviation from
the mean divided by the population standard deviation.
Equation 1
The shaded areas in each plot lie between the values of -z and +z: that are indicated to the left
and right of the curves. The numbers within the shaded areas are the percentage of the total area
under the curve that is included within these values of z. For example as shown in curve:
(a) 50% of the area under any Gaussian curve is located between -0.67σ and +0.67 σ.
(b) 80% of the total area lies between -1.28 σ and + 1.28 σ
(c) 90% lies between -1.64 σ and + 1.64 σ
(d) 95% lies between -1.96 σ and + 1.96 σ
(e) 99% lies between -2.58 σ and + 2.58 σ
Relationships such as these allow us to define a range of values around a measurement result
within which the true mean is likely to lie with a certain probability provided we have a reasonable
estimate of σ.
, For example, if we have a result x from a data set with a standard deviation of σ, we may assume
that 90 times out of 100, the true mean μ will fall in the interval x ± 1.64σ (see Figure c). The
probability is called the confidence level (CL).
In the example of Figure c, the confidence level is 90%, and the confidence interval is from -
1.64σ to +1.64 σ. The probability that a result is outside the confidence interval is often called
the significance level.
If we make a single measurement x from a distribution of known σ, we can say that the true mean
should lie in the interval x ± 𝑧σ with a probability dependent on z. This probability is 90% for z
= 1.64, 95% for z = 1.96, and 99% for z = 2.58, as shown in Figure c, d, and e.
We find a general expression for the confidence interval of the true mean based on measuring a
single value x by rearranging Equation 1. (Remember that z can take positive or negative values.)
Thus,
Confidence limit for 𝜇 = x ± 𝑧𝜎 Equation 2
Table 1
Rarely do we estimate the true mean from a single measurement, however. Instead, we use the
̅ of N measurements as a better estimate of μ. In this case, we replace x in
experimental mean 𝐗
̅ and σ with the standard error of the mean, σ/√𝑁. That is:
Equation 1 with 𝐗
𝒁𝛔
Confidence limit (CL) of 𝜇 = ̅
X ± Equation 3
√𝑵
Example 1
Glucose levels are routinely monitored in patients suffering from diabetes. The glucose
concentrations in a patient with mildly elevated glucose levels were determined in different months
by a spectrophotometric analytical method. The patient was placed on a low-sugar diet to reduce the
glucose levels. The following results were obtained during a study to determine the effectiveness of
the diet. Calculate a pooled estimate of the standard deviation for the method.
Time Glucose Concentration, mg/L
Month 1 1108, 1122, 1075, 1099, 1115, 1083, 1100
Month 2 992, 975, 1022, 1001, 991
