Time reversibility Consider an irreducible, positive recurrent CTMC, {X(t): t 0}, with a positive recurrent embedded MC. Suppose it starts in the stationary distribution i.e., P(x(0) + j) ~ Pi Fix some T > 0 and let the time-reversed CTMC be {Y(t) = X(T t): t = [0,T]}. = (Por.... What are the transition probabilities, and rates of leaving states for the reversed CTMC? -XCF) Y(F) t Time spent in state i~ Exp (Vi) Consider: Y(+) x(1) P (reversed CTMC stays in state i during interval of length s it starts in state i) P(original CTMC is in state i throughout interval [t s,t]|X(t) = i) = P(X (u) = i foru = [t-s =])+x(+) = i) / P(x(A)=i) = P(X (u) = i for ut (t-s,+])/ P(x(+) ==) ? = P(X (v) = i for ve [t-s, t] [ (t-s)=i) -| - -Vis (by def. of X(+)) P(x(t-s)=i) P(x(+7=1) -the time spent in i for Y(F) ~ Exp (v;) ~I ~ Problem 3 Based on the definition of a CTMC with rates v and transition probabilities pij, one can simulate a CTMC as follows (see also next week's notebook): From state i, (1) sample the time for the next jump from Exp(vi); (2) sample the new state from (Pi1, ..., Pin). Alternatively, we consider the following: From state i, (1) Sample Exp(qij) for all j such that Iij> 0; (2) compute their minimum, as the time for the next jump, (3) sample the new state as the value of j corresponding to this minimum (i.e. the argmin). Show that these two procedures are equivalent (hint: Use properties of Exp r.v. seen in class).