3. There are k players, with player i having valueI mage,I mage. In every period, two of the players play a game, while the otherI mage wait in an ordered line. The loser of a game joins the end of the line, and the winner then plays a new game against the player who is first in line. Whenever i and j play, i wins with probability .

(a) Define a Markov chain that is useful in analyzing this model.

(b) How many states does the Markov chain have?

(c) Give the transition probabilities of the chain.