Stochastic Processes Exam 1 UPDATED ACTUAL QUESTIONS AND CORRECT Answers

Stochastic Processes Exam 1 UPDATED ACTUAL QUESTIONS
AND CORRECT Answers


stochastic process a collection of rvs {x(t) : t ∈ T} = {X_t : t ∈ T}
where T ⊆ |R is the indexing set, usually "time": could be discrete or
continuous
- evolution of a "system" that varies over time


state space the set of all possible values (locations/states) for the stochastic process


Markov chain let {Xₙ : n ∈ |N} be a stochastic process on a finite or countable state
space S

, markov property a chain {Xₙ : n ∈ |N₀} has this if for every n ∈ |N₀, and any states s₁, s₂, s₃,
... , sₙ, sₙ₊₁ ∈ S
|P(Xₙ₊₁ = sₙ₊₁ | X₀ = s₀, X₁ = s₁, ... , Xₙ = sₙ) = |P(xₙ₊₁ = sₙ₊₁|Xₙ = sₙ)


- future behavior only depends on the "current" state and is independent
of the past


markov chain a discrete-time process that satisfies the Markov property


initial distribution of {Xₙ} is the probability distribution of the rv x₀. Use π₀ to denote the
discrete density (pdf) of x₀


i.e. π₀(x) = |P(X₀=x) ∀ x ∈ S
note: π₀(x) ≥ 0 ∀ x∈S and ∑_{x ∈ S} π₀(x) = 1


one-step transition probabilities the conditional probabilities
|P(Xₙ₊₁ = y | Xₙ = x) ∀ x, y ∈ S, n ∈ |N₀


we assume that |P(Xₙ₊₁ = y | Xₙ = x) = |P(X₁ = y | X₀ = x)
(assuming that {Xₙ} has the time-homogenous transition probabilities,
which we do)


transition function P(x, y) = |P(X₁ = y | X₀ = x) = |P(Xₙ₊₁ = y | Xₙ = x)


simple symmetric random walk on /Z let X₀ ∈ /Z be the initial position of a particle. suppose the particle moves
one step to the left or right with equal probability, independent of the
other steps


markov requirements - state space S is an most countably-∞
- the initial distribution π₀(x)= |P(X₀ = x) ∀ x ∈ S
- the transition function: P(x, y) = = |P(X₁ = y | X₀ = x) = |P(Xₙ₊₁ = y | Xₙ
= x)


simple random walk on S = /Z of the form Xₙ = X₀ + ∑_{k=1}^n /xi_k where \xi₁, \xi₂, ... are iid with
|P(\xi = 1) = p,
|P(\xi = 0) = r,
|P(\xi = -1) = q,
p, q, r ≥ 0,
w/ p+q+r = 1


is is SSRW when p = q


transition function of the SRW P(x, y) = {p if y = x +1, r if y = x, q if y = x-1, 0 otherwise}


Ehrenfest chain suppose we have 2 urns labeled A and B and we have d balls numbered
1,..., d.
- put each ball into an urn at random
- let X₀ = # of balls in urn A initially. Note X₀ ~ Bin(d, 1/2)
- at each step, pick an integer 1, ..., d at random, move the
corresponding ball to the other urn
- let Xₙ = # of balls in urn A after the nth step
- state space: S= {0, 1, ..., d}


transition function of the Ehrenfest chain P(x, y) = {x/d if y = x - 1, 1 - x/d if y = x + 1, 0 otherwise}