Here is a model of some kinds of learning The learner starts in an undecided state sU. Eventually the learner has to decide to do either response A (that is, end in state sA) or response B (ending in sB). However, the learner doesn't jump right from undecided to sure that A is the correct thing to do (or B). Instead, the learner spends some time in a "tentative-A" state, or a "tentative-B" state, trying the response out (denoted here tA and tB). Imagine that once the learner has decided, it is final, so once in sA or sB, the learner stays there. For the other state changes, we can posit transitions with probability p in either direction.
(a) Construct the transition matrix.
(b) Take p = 0.25 and take the initial vector to be 1 at sU. Run this for five steps. What is the chance of ending up at sA?
(c) Do the same for p = 0.20.
(d) Graph p versus the chance of ending at sA. Is there a threshold value for p, above which the learner is almost sure not to take longer than five steps?