4. Let P and Q be transition probability matrices on statesI mage, with respective transition probabilities and . Consider processes andI mage defined as follows:

(a) Image. A coin that comes up heads with probability p is then flipped. If the coin lands heads, the subsequent states , are obtained by using the transition probability matrix P; if it lands tails, the subsequent states , are obtained by using the transition probability matrix Q. (In other words, if the coin lands heads (tails) then the sequence of states is a Markov chain with transition probability matrix P (Q).) Is a Markov chain. If it is, give its transition probabilities. If it is not, tell why not.

(b) Image. If the current state is i, then the next state is determined by first flipping a coin that comes up heads with probability p. If the coin lands heads then the next state is j with probability ; if it lands tails then the next state is j with probability . Is Image a Markov chain. If it is, give its transition probabilities. If it is not, tell why not.