Two gamblers, call them Gambler A and Gambler B, play repeatedly. In each round, A wins 1 dollar with probability p or loses 1 dollar with probability q = 1 −p (thus, equivalently, in each round B wins 1 dollar with probability q = 1 −p and loses 1 dollar with probability p). We assume different rounds are independent. Suppose that initially A has i dollars and B has N −i dollars. The game ends when one of the gamblers runs out of money (in which case the other gambler will have N dollars). Our goal is to find pi, the probability that A wins the game given that he has initially i dollars.

a. Define a Markov chain as follows: The chain is in state i if the Gambler A has i dollars. Here, the state space is S = {0, 1,⋯,N}. Draw the state transition diagram of this chain.

b. Let ai be the probability of absorption to state N (the probability that A wins) given that X₀ = i. Show that

c. Show that

d. Find ai for any i ∈ {0, 1, 2,⋯,N}. Consider two cases: p = 1/2 and p ≠ 1/2.

Transcribed Image Text:

ao = aN - = ai+1 ai = = 0, 1, 9 -(a; -ai-1), Р for i=1,2,, N - 1.