Probability and Stochastic Processes

8 A. K. MAJEE

2. Random variables and their distributions
Frequently, when an experiment is performed, we are interested mainly in some function of
the outcome as opposed to the actual outcome itself. For instance, in tossing dice, we are often
interested in the sum of the two dice and are not really concerned about the separate values of
each die.
Definition 2.1 (Random variable). Let (Ω, F, P) be a probability space. A real-valued
function X : Ω → R is said to be random variable if for any B ∈ B(R), X −1 (B) ∈ F.
Example 2.1. Let (Ω, F, P) be a probability space and A ∈ F. Define a function X : Ω → R by
(
1 if ω ∈ A
X(ω) =
0 if ω ∈/A

Then X is a random variable. Indeed, for any x ∈ R, we see that

∅ if x < 0

{ω ∈ Ω : X(ω) ≤ x} = A{ if 0 ≤ x < 1

Ω if x ≥ 1.


The function X is called an indicator function of A and often denoted by χA or IA .
Example 2.2. Let Ω = {0, 1, 2} and F = {∅, {0}, {1, 2}, Ω}. Then (Ω, F) is a measurable space.
Define a function X on Ω by X(i) = i for i ∈ Ω. Then X is NOT a random variable. Indeed,
for any x ∈ R, we see that


∅ if x < 0

{0} if 0 ≤ x < 1
{i ∈ Ω : X(i) ≤ x} =

{0, 1} if 1 ≤ x < 2

Ω if x ≥ 2.


Since {0, 1} ∈
/ F, by definition X is NOT a random variable.
For any B ∈ B(R), we are interested in P(X ∈ B). Let X be a random variable defined on a
given probability space (Ω, F, P). Define a function PX on B(R) via
PX (B) := P(X ∈ B)
One can easily check that PX is a probability measure on (R, B(R)). This is called distribution
of X.
Definition 2.2 (Distribution function/ Cumulative distribution function (cdf ) ). Let
X be a random variable defined on a probability space (Ω, F, P). Then the function FX : R →
[0, 1] defined by
FX (x) = P(X ≤ x) = PX ((−∞, x])
is called the distribution function or cumulative distribution function (cdf) of the random vari-
able X.
Example 2.3. A fair coin is tossed twice: Ω = {HH, HT, T H, T T }. Take F = P(Ω) and
define a probability measure P(A) = |A|
4 for any A ∈ F. For ω ∈ Ω, let X(ω) be the number of
heads so that
X(HH) = 2, X(HT ) = X(T H) = 1, X(T T ) = 0.

, PROBABILITY AND STOCHASTIC PROCESS 9

Then X is a random variable. Indeed for any x ∈ R, one has


 ∅ for x < 0

{T T } for 0 ≤ x < 1
{ω : X(ω) ≤ x} =

 {T T, HT, T H} for 1 ≤ x < 2

Ω for x ≥ 2.


The distribution function FX of X is then given by


0 for x<0
1

for 0≤x<1
FX (x) = P(ω : X(ω) ≤ x) = 43
for 1≤x<2
4


1 for x ≥ 2.


Next we discuss some essential properties of distribution function.
Lemma 2.1. The distribution function FX satisfies the following properties:
a) Nondecreasing: if x < y, then FX (x) ≤ FX (y).
b) Right continuity: F is right-continuous i.e., FX (x + h) → FX (x) as h ↓ 0.
c) Left limit: FX (·) has left limit and FX (x−) = P(X < x).
d) limx→∞ FX (x) = 1 and limx→−∞ FX (x) = 0.
e) P(x < X ≤ y) = FX (y) − FX (x).
f ) P(x ≤ X ≤ y) = FX (y) − FX (x−).
g) P(x ≤ X < y) = FX (y−) − FX (x−), P(x < X < y) = FX (y−) − FX (x).
h) P(X = x) = FX (x) − FX (x−).
Proof. Proof of a): For x < y, we have {X ≤ x} ⊆ {X ≤ y} and hence FX (x) ≤ FX (y).

Proof of b): Since F is nondecreasing, it is sufficient to show that FX (xn ) → FX (x) for any
sequence of numbers xn ↓ x with x1 ≥ x2 ≥ . . . xn > x. Define An := {X ≤ xn }. Then
An+1 ⊆ An and ∩∞n=1 An = {X ≤ x}. Hence by using the property of P, we get

FX (x) = P(X ≤ x) = P(∩∞
n=1 An ) = lim P(An ) = lim FX (xn ).
n→∞ n→∞

Proof of c): Let x ∈ R be fixed. Let {xn } be such that x1 ≤ x2 ≤ . . . < x and limn→∞xn =x .
Take An = {X ≤ xn }. Then An ⊆ An+1 and ∩∞ n=1 An = {X < x}. Thus, we have

P(X < x) = P(∩∞
n=1 An ) = lim P(An ) = lim FX (xn ).
n→∞ n→∞

Thus, FX (·) has left limit and FX (x−) = P(X < x).
Proof of d): Observe that {X ≤ n} ⊆ {X ≤ (n + 1)} and Ω = ∪∞
n=1 {X ≤ n}. Hence by using
the property of P, we get
1 = P(Ω) = P(∪∞
n=1 {X ≤ n}) = lim P({X ≤ n}) = lim FX (n) = lim FX (x).
n→∞ n→∞ x→∞

For the second part take An = {X ≤ −n}. Then An is decreasing and ∩∞
n=1 An = ∅. Hence one
has
0 = P(∅) = P(∩∞
n=1 An ) = lim P(An ) = lim FX (−n) = lim FX (x).
n→∞ n→∞ x→−∞

Proof of e): Since PX is a probability measure on (R, B(R)) , by using the property of a
measure that for any B ⊆ A, there holds PX (A) − PX (B) = PX (A \ B) and the definition
FX (x) = PX ((−∞, x]), we see that
FX (y) − FX (x) = PX ((x, y]) = P(x < X ≤ y).