Introduction to Stochastic Processes with R Complete Solutions Manual

Instructor’s Solutions Manual

for


Introduction to Stochastic Processes with R


by Robert P. Dobrow




March 6, 2016




1

,Chapter 1
1.1 For the following scenarios identify a stochastic process {Xt , t ∈ I}, describing (i) Xt
in context, (ii) state space, and (iii) index set. State whether the state space and index
set are discrete or continuous.

b) Xt is the student’s status at the end of year t. State space (discrete): S =
{Drop Out, Frosh, Sophomore, Junior, Senior, Graduate}. Index set (discrete): I =
{0, 1, 2, . . .}.
c) Xt is the magnitude for an earthquake which occurs at time t. State space (contin-
uous): (0, 10). Index set (continuous): [0, ∞).
d) Xt is the circumference of the tree at location t. State space (continuous): (0, ∞).
Index set (continuous): [0, 2] × [0, 2]; or the x − y coordinates of a location in the
arboretum.
e) Xt is the arrival time of student t. State space (continuous): [0, 60]. Index set
(discrete): {1, 2, . . . , 30}.
f) Xt is the order of the deck of cards after t shuffles. State space (discrete): Set of
all orderings of the deck (52! elements). Index set (discrete): {0, 1, 2, . . .}.

1.2 A regional insurance company insures homeowners against flood damage. Half of their
policyholders are in Florida, 30% in Louisiana, and 20% in Texas.

a) Let A be the event that a claim is filed for flood damage. Then

P (A) = P (A|F )P (F ) + P (A|L)P (L) + P (A|T )P (T )
= (0.03)(0.50) + (0.015)(0.30) + (0.02)(0.20) = 0.0235.


b) P (T |A) = P (A|T )P (T )/P (A) = (0.02)(0.20)/0.0235 = 0.17.

1.3 Let B1 , . . . , Bk be a partition of the sample space. For events A and C, prove the law
of total probability for conditional probability.


k k   
X X P (A ∩ Bi ∩ C) P (Bi ∩ C)
P (A|Bi ∩ C)P (Bi |C) =
i=1 i=1
P (Bi ∩ C) P (C)
k k
X P (A ∩ Bi ∩ C) 1 X
= = P (A ∩ Bi ∩ C)
i=1
P (C) P (C) i=1
P (A ∩ C)
= = P (A|C).
P (C)


1.4 See Exercise 1.2. Among policyholders who live within 5 miles of the coast, 75% live
in Florida, 20% live in Louisiana, and 5% live in Texas. Suppose a policyholder lives
within 5 miles of the coast. Use the law of total probability for conditional probability
to find the chance they will file a claim for flood damage next year.


2

, Let A be the event that a claim is filed. Let C be the event that a claimant lives
within five miles of the coast. Then

P (A|C) = P (A|F, C)P (F |C) + P (A|L, C)P (L|C) + P (A|T, C)P (T |C)
= (0.10)(0.75) + (0.06)(0.20) + (0.06)(0.05) = 0.09.



1.5 Two fair, six-sided dice are rolled. Let X1 , X2 be the outcomes of the first and second
die, respectively.

a) Uniform on {1, 2, 3, 4, 5, 6}.
b) Uniform on {2, 3, 4, 5, 6}.
1.6 Bob has n coins in his pocket. One is two-headed, the rest are fair. A coin is picked
at random, flipped, and shows heads.

Let A be the event that the coin is two-headed.
P (H|A)P (A) (1)(1/n) 2
P (A|H) = c c
= = .
P (H|A)P (A) + P (H|A )P (A ) (1)(1/n) + (1/2)((n − 1)/n) n+1


1.7 A rat is trapped in a maze with three doors and some hidden cheese.

Let X denote the time until the rat finds the cheese. Let 1, 2, and 3 denote each door,
respectively. Then

E(X) = E(X|1)P (1) + E(X|2)P (2) + E(X|3)P (3)
1 1 1 2
= (2 + E(X)) + (3 + E(X)) + (1) = 2 + E(X) .
3 3 3 3
Thus, E(X) = 6 minutes.
1.8 A bag contains 1 red, 3 green, and 5 yellow balls. A sample of four balls is picked. Let
G be the number of green balls in the sample. Let Y be the number of yellow balls in
the sample.

a)
P (G = 1|Y = 2) = P (G = 2|Y = 2) = 1/2.

b) The conditional distribution of G is binomial with n = 2 and p = 3/9 = 1/3.
 
2
P (G = k|Y = 2) = (1/3)k (2/3)2−k , for k = 0, 1, 2.
k


1.9 Suppose X is uniformly distributed on {1, 2, 3, 4}. If X = x, then Y is uniformly
distributed on {1, . . . , x}.

a) P (Y = 2|X = 2) = 1/2.


3

, b)
P (Y = 2) = P (Y = 2|X = 2)P (X = 2) + P (Y = 2|X = 3)P (X = 3)
+ P (Y = 2|X = 4)P (X = 4) = (1/2)(1/4) + (1/3)(1/4) + (1/4)(1/4)
= 13/48.

c) P (X = 2|Y = 2) = P (X = 2, Y = 2)/P (Y = 2) = (1/8)/(13/48) = 6/13.
d) P (X = 2) = 1/4.
e) P (X = 2, Y = 2) = P (Y = 2|X = 2)P (X = 2) = (1/2)(1/4) = 1/8.
1.10 A die is rolled until a 3 occurs. By conditioning on the outcome of the first roll, find
the probability that an even number of rolls is needed.

Let A be the event that an even number of rolls is needed. Let B be the event that a
3 occurs on the first roll.
P (A) = P (A|B)P (B) + P (A|B c )P (B c ) = (0)(1/6) + (1 − P (A))(5/6)
gives P (A) = 5/11.
1.11 Consider gambler’s ruin where at each wager, the gambler wins with probability p and
loses with probability q = 1 − p. The gambler stops when reaching $n or losing all
their money. If the gambler starts with x, with 0 < x < n, find the probability of
eventual ruin.

Let xk be the probability of reaching n when the gambler’s fortune is k. Then
xk = xk+1 p + xk−1 q, for 1 ≤ k ≤ n − 1,
with x0 = 0 and xn = 1, which gives
q
xk+1 − xk = (xk − xk−1 ) , for 1 ≤ k ≤ n − 1.
p
It follows that
xk − xk−1 = · · · = (x1 − x0 )(q/p)k−1 = x1 (q/p)k−1 , for all k.
Pk
This gives xk − x1 = i=2 x1 (q/p)i−1 . For p 6= q,
k
X 1 − (q/p)k
xk = x1 (q/p)i−1 = x1 .
i=1
1 − q/p

For k = n, this gives
1 − (q/p)n
1 = xn = x1 .
1 − q/p
Thus x1 = (1 − q/p)/(1 − (q/p)n ), which gives
1 − (q/p)k
xk = , for k = 0, . . . , n.
1 − (q/p)n

For p = q = 1/2, xk = k/n, for k = 0, 1, . . . , n.


4