ST2133 - Distribution Theory

DISTRIBUTION THEORY
ADVANCED
CLASS
STATISTICS

Ad Stats: Dist Theory


Name Description Logical Statement

The σ -algebra with the
largest number of members is
Let Ψbe a set and let Bbe a collection of subsets of Ψ.
given by including every
Bis called a σ -algebra. i) ∅ ∈ B, i.e. the empty set is a
Definition of subset of Ψ. B = {A :
member of Bii) If A ∈ B, then Ac ∈ B, i.e. Bis closed
$\sigma-algebra$ A ⊆ Ψ} This is referred to
under complementation iii) If A1 , A2 , ... ∈ B, then
as the power set of Ψ,
​ ​




∪Ai ∈ B, i.e. Bis closed under countable unions
sometimes written P (Ψ)or
​




{0, 1} Ψ 
Given a measurable space (Ψ, B), a measure on
(Ψ, B)is a function, m, where m : B → ℜ+ i) Measures allow us to express
Definition of
m(A) ≥ 0 for all A ∈ Bii) m(∅) = 0 iii) If the size of sets The size of
measure
A1 , A2 , ... ∈ Bare mutually exclusive then:
​ ​ the power set is 2 n(Ψ) 
m(∪Ai ) = Σm(Ai ) ​ ​




Probability of a union of disjoint events is the sum of the
Boole Inequality individual probabilities. When the events are not disjoint,
then P (∪Ai ) ≤ ΣP (Ai ) ​ ​




The number of possible ordered sequences, of k objects
Permutations selected without replacement from nobjects n Pk = ​




n!
(n−k)!
 ​




The number of unordered sequences, of k objects
also known as the binomial
Combinations selected without replacement from nobjects n Ck = ​




coefficient
(nk ) = k!(n−k)!
​
n!
 ​




Chain Rule of
P (A1 ∩ A2 ∩ A3 ) = P (A1 ∣A2 ∩ A3 )P (A2 ∩ A3 ) P (∩Aj ) =
 0 then:
Conditional
​ ​ ​ ​ ​ ​ ​ ​ ​




= P (A1 ∣A2 ∩ A3 )P (A2 ∣A3 )P (A3 ) P (∩Aj ) = ΠP (Aj ∣ ∩ Ai )
Probabilities
​ ​ ​ ​ ​ ​ ​ ​ ​




1. Fx is a non-decreasing function, i.e. if x < ythen
Properties of
​




Fx (x) ≤ Fx (y)2. limx→−∞ Fx (x) = P (X = x) = FX (x) −
Distribution
​ ​ ​ ​ ​




0 and limx→∞ Fx (x) = 1 3. Fx is right continuous, i.e. FX (x− )
Functions
​ ​ ​ ​




Fx (x+ ) = Fx (x)for all x ∈ R
​ ​




X ∼ Degenerate(a)Concentrates all probability at a
Degenerate 1 f or x=a
single point. p X (x) = { 0 otherwise Distribution function ​ ​
moment generating function:

, Name Description Logical Statement
random variable, such that, xdenotes the number of
favourable outcomes with a constant probability of
n
success π . X ∼ Bin(n, π)p X (x) = (x)π x (1 − ​ ​




π)n−x for x ≥ 0 FX (x) = Σxi=0 (ni )π i (1 − π)n−i for
​ ​ ​




0 ≤ x < n
Consider an experiment of events with only two possible
outcomes. Let X be the random variable, such that, x
denotes the number of events for the first favourable
Geometric moment generating function:
outcome. E.g. Flipping of a coin x = 4 times. There are 3 πe t
Distribution v1 MX (t) = 1−e t (1−π)

(4-1) tails (unfavourable) before getting the first head.
​ ​




X ∼ Geo1 (π)pX (x) = (1 − π)x−1 π for X ≥ 1 
​ ​




FX (x) = 1 − (1 − π)x for X ≥ 1 
​




Consider an experiment of events with only two possible
outcomes. Let X be the random variable, such that, x
denotes the number of failures before the first favourable
Geometric moment generating function:
outcome. E.g. Getting x = 3 tails on the flip of a coin π
Distribution v2 MX (t) = 1−et (1−π) 
∼ Geo0 (π)pX (x) =
​ ​




before getting the first head. X ​ ​




(1 − π)x π for X ≥ 0 FX (x) ​ = 1 − (1 − π)x+1 for
X ≥ 0
Extends the Geometric distribution to X number of
independent events until the r number of favourable
Negative outcomes with a constant probability of success π . In
th
moment generating function:
Binomial order to have r favourable outcomes on the X trial, we (πe t ) r
MX (t) = 
Distribution v1 must have r − 1 outcomes favourable in the first x − 1  [1−(1−π)e t ]r
​ ​




x−1
trials. X ∼ N eg. Bin(r, π)p X (x) = ( r−1 )π r (1 − ​ ​




π) x−r
for X ≥ r
Negative X − N eg. Bin∗ (r, π), where X is the number of
moment generating function:
Binomial failures before the r th successes p X (x) = πr
MX (t) = [1−(1−π)e t ]r 
​




(x+r−1 )π r (1 − π)x for X ≥ 0 
​ ​




Distribution v2 r−1
​




Newton’s
Binomial Σ(mi )Z i = (Z + 1)m 
​




Theorem

Let X be the random variable, such that, xdenotes the
🌟 Poisson number of rare occurrences of events over a fixed period.
e −λ λx
moment generating function:
t
Distribution X ∼ P ois(λ)pX (x) = ​




x!
for
​ x ≥ 0 FX (x) = ​
MX (t) = eλ(e −1) 
​




−λ i
Σx e i!λ for x ≥ 0 ​




Continuous 1
X ∼ U [a, b]fX (x) = b−a
for a ≤ x ≤ bFX (x) = moment generating function:
Uniform
​ ​ ​




x−a e bt −e at
for a ≤ x < b MX (t) = (b−a)t

Distribution b−a
​ ​ ​




X ∼ Exp(λ)fX (x) = λe−λx for x ≥ 0 FX (x) =
Exponential moment generating function:
​ ​




1 − e−λx for x ≥ 0 https://statproofbook.github.io/P/exp- λ
Distribution MX (t) = λ−t
​
 ​




cdf.html
1
Normal X ∼ N (μ, σ 2 )fX (x) = ​




2πσ 2
exp[− 2σ1 2 (x
​ ​ − μ)2 ] moment generating function:

​


2 2