Assignment overview. Our goal is to perform an analysis of long-term (or steady-state) performance of a Markov Chain. Suppose you are given a transition matrix P. Your task is to estimate the long-run proportions by observation. Of cause, you can achieve the same result using analytical methods. The purpose here is to illustrate the principle, so that you can then apply it to more complex systems, where analytical methods are intractable. The direct approach to obtain long-run performance profile is to perform a simulation, i.e., trace the transitions of the Markov chain for a certain number of iterations, until the long-run behavior has been established, counting the proportion of time that the system spends in each state. According to the theoretical result, if this simulation is performed for a long enough time, these proportions will converge to the long-run probabilities that you can find by solving the system of equations as discussed in class. In your assignment you are asked to first, perform a simulation estimating these probabilities, and then analyze the number of transitions that is required for you to achieve a reasonable accuracy. You can estimate the long-run proportions by tracking the state of the system over a long period of time and counting the number of times that the system enters each state.

Directions

As the underlying model consider Bonus-Malus insurance policy described in class and in your book (example 4.7). Each of you should use their own, individual value of the Poisson parameter . To obtain your , follow the link http://www.wolframalpha.com/input/?i=random+ number+between+0.1+and+2.0 using the value in the Result filed.

Define transition matrix P. For example, in Python a 3 3 matrix can be defined by: p = [ [ . 4 , . 4 , . 2 ] , [ . 0 , . 5 , . 5 ] , [ . 0 , . 4 , . 6 ] ] Here the first row is (0.4, 0.4, 0.2), second row is (0, 0.5, 0.5), etc. In your case, you will need to define a 4 4 matrix. To obtain the specific values for matrix elements, you can either precompute the transition probabilities according to the Poisson distribution or do it in Python according to the Poisson formula (the latter is the preferred method), see example 4.7 in the book and class discussion for details. Note that in Python lambda is a reserved word, so you will need to use a different name for the corresponding variable.

Define an array of state counters (I will call it scnt); the number of simulation steps that you will use (I will call it nsteps); and the current state (state). Originally set 1 s c n t = [ 0 , 0 , 0 , 0 ] n s t e p s = 100 s t a t e = 0 The array will be used to count the amount of time that the systems spends in each state, if observed for nsteps iterations. Variable state keeps track of the current state.

Now, perform transitions nsteps times according to the transition matrix, recording the progression, i.e., repeat the following nsteps times. increment the state counter that corresponds to the current state, i.e., record the current location perform a transition. The next state is determined by the current state (state) and the corresponding row of the transition matrix. There are different ways to do that, one of which is to use a uniform random variable. draw a realization of a uniform random variable (function u = random.random()) in Python for the row of the transition matrix that corresponds to the current state, add the transition probabilities one by one from left to right until the sum exceeds the value of u. The first time when that happens determines the next state

Compute the long run proportions by dividing each scnt by nsteps.

Analyze how the values you obtain change with the value of nsteps. Identify the value of nsteps that is enough to determine the long-run proportions with at least two digits of accuracy.