Consider the following gambler€™s ruin problem. A gambler bets $1 on each play of a game. Each time, he has a probability p of winning and probability q = 1 p of losing the dollar bet. He will continue to play until he goes broke or nets a fortune of T dollars. Let Xn denote the number of dollars possessed by the gambler after the nth play of the game. Then

{Xn} is a Markov chain. The gambler starts with X0 dollars, where X0 is a positive integer less than T.
(a) Construct the (one-step) transition matrix of the Markov chain.
(b) Find the classes of the Markov chain.
(c) Let T = 3 and p = 0.3. Using the notation of Sec. 29.7, find f10, f1T, f20, f2T.
(d) Let T = 3 and p = 0.7. Find f10, f1T, f20, f2T.

Transcribed Image Text:

Xn x with probability p with probability 1-p for 0 XT 0-x x, T Ant-1 for X, 0, or T. n+1