3.8. Markov processes Brownian motion is a particular example of a general class of stochastic processes known as Markov processes, characterized by a transition probability w(i→j) = wij between the discrete states {A} = {a1,a2,a3, . . . ,an} of a stochastic variable A (wij ≥ 0 andnj

=1wij =

1). These transitions take place at discrete time steps tn = n. Denoting by Pi(n) the probability to be in the ith state at the time n, it satisfies the recursive equation Pi(n+1) =

n



j=1 wij Pj(n).

Using a matrix formalism, it can be expressed as P(n+1) = WP(n).

a. Prove that the eigenvalues of the matrix W satisfy the condition | λi |≤ 1.

b. Show that the system reaches an equilibrium distribution Pi(∞) for t→∞that is independent from the initial condition if and only if the matrix W has only one eigenvalue of modulus 1.

c. Assuming that the conditions of point b are satisfied, prove that