ST3189 - Block 4 (Bayesian Inference)

BLOCK 4 - BAYESIAN INFERENCE
MACHINE
CLASS
LEARNING



💡 KEY CONCEPTS:




RECALL NOTES
Introduction The study of machine learning revolves heavi
prediction

Two main interpretations of probability

Frequentist

Bayesian



📌
Frequentist Inference
“probability is based on frequencies of e



Maximum Likelihood Estimation (MLE)

Fixed point estimate

There exist a population parameter θp that h
​




📌
Bayesian Inference
“probability is based on our knowledge of



Incorporates prior knowledge about event int

Bayes’ Theorem

, θ^ = E(θ∣x) = ∫ θ π(θ∣x ​




θ

Conditional expectations recap




Posterior Median, θ^ = q

P (θ ≤ q∣y) = P (θ ≥ q∣y)

Posterior Mode, θ^ = q
a.k.a Maximum A-Posteriori (MAP)

π(θ∣y) ≤ π(q∣y) for al

∂ℓ
θM LE ⇒ =0
∂θ
​ ​




Where:

ℓis the log-likelihood function


Computation of Credible intervals

The range of a particular posterior proba

Bayes Factor

Ratio of two competing statistical models
marginal likelihood

In layman terms, compare the likelihood o
M2 
​




P (y∣M1 ) P (M1 ∣y)/P (M2 ∣y)
BF1,2 = =
​ ​ ​




P (y∣M2 ) P (M1 )/P (M2 )
​ ​ ​




​ ​ ​




Reject H1 if BF1,2 < 1
​ ​




1

Since the likelihood (ratio) of H
H2 ​