UNIVERSITY OF SOUTH AFRICA
College of Science, Engineering and Technology
PHY3703
Statistical and Thermal Physics
Formative Assessment 2
Prescribed Textbook: Harvey Gould and Jan Tobochnik,
Statistical and Thermal Physics with Computer Applications,
Princeton University Press, 2010
Problems Covered: 3.20, 3.37, 4.10, 4.21
Submission Date: As per myUnisa
Submitted in partial fulfilment of the requirements for PHY3703
, Statistical and Thermal Physics PHY3703 – Assessment 2
Problem 3.20 Uncertainty (Entropy)
The statistical uncertainty, or information entropy, is defined as:
X
S=− Pn ln Pn
n
where Pn is the probability of outcome n.
1
(a) Uncertainty for P1 = P2 = 2
When both outcomes are equally likely, the uncertainty is:
1 1 1 1 1
S=− ln + ln = − ln = ln 2
2 2 2 2 2
S = ln 2 ≈ 0.693
This is the maximum possible uncertainty for a two-outcome system. When both
outcomes are equally probable, we have the least information about what will happen.
(b) Uncertainty for P1 = 51 , P2 = 45 , and comparison with (a)
1 1 4 4
S=− ln + ln
5 5 5 5
Computing each term:
1 1
ln = 0.2 × (−1.6094) = −0.3219
5 5
4 4
ln = 0.8 × (−0.2231) = −0.1785
5 5
S = −[(−0.3219) + (−0.1785)] = 0.500
S ≈ 0.500 < ln 2 ≈ 0.693
The uncertainty is lower here than in part (a). Because one outcome is much more
probable than the other, the system is more predictable – there is less surprise in the
result. Unequal probabilities always reduce entropy compared to the uniform case.
(c) Comparing the uncertainty of the third and fourth experiments
The core principle here is that entropy is maximised when all outcomes are equally prob-
able and decreases as the probability distribution becomes more skewed.
So, comparing any two experiments with two outcomes each:
The experiment whose outcome probabilities are closer to 1
2
each will have the higher
uncertainty.
1
College of Science, Engineering and Technology
PHY3703
Statistical and Thermal Physics
Formative Assessment 2
Prescribed Textbook: Harvey Gould and Jan Tobochnik,
Statistical and Thermal Physics with Computer Applications,
Princeton University Press, 2010
Problems Covered: 3.20, 3.37, 4.10, 4.21
Submission Date: As per myUnisa
Submitted in partial fulfilment of the requirements for PHY3703
, Statistical and Thermal Physics PHY3703 – Assessment 2
Problem 3.20 Uncertainty (Entropy)
The statistical uncertainty, or information entropy, is defined as:
X
S=− Pn ln Pn
n
where Pn is the probability of outcome n.
1
(a) Uncertainty for P1 = P2 = 2
When both outcomes are equally likely, the uncertainty is:
1 1 1 1 1
S=− ln + ln = − ln = ln 2
2 2 2 2 2
S = ln 2 ≈ 0.693
This is the maximum possible uncertainty for a two-outcome system. When both
outcomes are equally probable, we have the least information about what will happen.
(b) Uncertainty for P1 = 51 , P2 = 45 , and comparison with (a)
1 1 4 4
S=− ln + ln
5 5 5 5
Computing each term:
1 1
ln = 0.2 × (−1.6094) = −0.3219
5 5
4 4
ln = 0.8 × (−0.2231) = −0.1785
5 5
S = −[(−0.3219) + (−0.1785)] = 0.500
S ≈ 0.500 < ln 2 ≈ 0.693
The uncertainty is lower here than in part (a). Because one outcome is much more
probable than the other, the system is more predictable – there is less surprise in the
result. Unequal probabilities always reduce entropy compared to the uniform case.
(c) Comparing the uncertainty of the third and fourth experiments
The core principle here is that entropy is maximised when all outcomes are equally prob-
able and decreases as the probability distribution becomes more skewed.
So, comparing any two experiments with two outcomes each:
The experiment whose outcome probabilities are closer to 1
2
each will have the higher
uncertainty.
1
