Summary Probability Theory: A Comprehensive Course With Detailed Examples, Applications and Solved Exercises

Probability Theory
A Comprehensive Course
With Detailed Examples, Applications and Solved Exercises




Author: Tichicht med
Version: 2.0 – Professional Edition

© 2026 – All rights reserved


Over 20 pages of rigorous content, no wasted space.

,2

,Contents

1 Foundations of Probability 5
1.1 Sample Space and Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.1 Set Operations on Events . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Axioms of Probability (Kolmogorov) . . . . . . . . . . . . . . . . . . . . . . 5
1.3 Counting Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Conditional Probability and Independence 7
2.1 Conditional Probability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.2 Law of Total Probability and Bayes’ Theorem . . . . . . . . . . . . . . . . . 8
2.3 Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Random Variables 9
3.1 Definition and Types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.1.1 Probability Mass Function (PMF) – Discrete . . . . . . . . . . . . . . 9
3.1.2 Probability Density Function (PDF) – Continuous . . . . . . . . . . . 9
3.1.3 Cumulative Distribution Function (CDF) . . . . . . . . . . . . . . . . 10
3.2 Transformations of Random Variables . . . . . . . . . . . . . . . . . . . . . . 10

4 Expectation and Variance 11
4.1 Expected Value . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
4.2 Variance and Standard Deviation . . . . . . . . . . . . . . . . . . . . . . . . 11
4.3 Covariance and Correlation . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

5 Common Probability Distributions 13
5.1 Discrete Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.1.1 Bernoulli(p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.1.2 Binomial(n, p) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.1.3 Poisson(λ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.2 Continuous Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3

,CONTENTS 4


5.2.1 Uniform(a, b) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.2.2 Exponential(λ) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
5.2.3 Normal (Gaussian) N (µ, σ ) . . . . . . . . . . . . . . . . . . . . . . .
2
14
5.2.4 Gamma and Chi-square . . . . . . . . . . . . . . . . . . . . . . . . . 14

6 Limit Theorems 15
6.1 Law of Large Numbers (LLN) . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.2 Central Limit Theorem (CLT) . . . . . . . . . . . . . . . . . . . . . . . . . . 15
6.3 Markov Chains (Brief Introduction) . . . . . . . . . . . . . . . . . . . . . . . 16

Exercises with Detailed Solutions 17

Final Notes 21

, Chapter 1

Foundations of Probability

1.1 Sample Space and Events
Probability is the mathematical framework for quantifying uncertainty. Every random ex-
periment is described by a sample space.

Definition 1.1: Sample Space and Events
The sample space Ω is the set of all possible outcomes of a random experiment. An
event is any subset A ⊆ Ω. Events are often denoted by capital letters A, B, C. The
empty set ∅ is the impossible event, and Ω itself is the certain event.

Example 1.1: Tossing a coin and rolling a die

Toss a fair coin: Ω = {H, T }. Roll a six-sided die: Ω = {1, 2, 3, 4, 5, 6}. For the die,
event A = “even number” = {2, 4, 6}.



1.1.1 Set Operations on Events
Since events are sets, we use:
• Union A ∪ B: outcome in A or B (or both).
• Intersection A ∩ B: outcome in both A and B.
• Complement Ac = Ω \ A: outcome not in A.
• Difference A \ B = A ∩ B c .
Two events are mutually exclusive if A ∩ B = ∅.


1.2 Axioms of Probability (Kolmogorov)
A probability measure P assigns a number P(A) to each event A, obeying:


5