In this assignment I would like you to implement a certain computational task using a programming language of your choice. For everyone in INSY 3010 I would strongly suggest that you use Python. The following instructions are written with Python in mind. Python must be used for this assignment.

Assignment overview. Our goal is to perform an analysis of long-term performance of a Markov Chain with absorbing states. Suppose you are given a transition matrix P. Your task is to estimate the probabilities that the Markov chain converges to each specific absorbing state. Of cause, you can achieve the same result using analytical methods, that we will discuss in class if we have time. The purpose here is to illustrate the principle, so that you can then apply it to more complex systems, where analytical methods are intractable. The direct approach to obtain long-run performance profile is to perform a simulation, i.e., trace the transitions of the Markov chain for a certain number of iterations, until absorption, and then repeat it multiple times (this is known as Monte-Carlo simulation). If you repeat the experiment enough time, observed proportion of time that the Markov chain gets absorbed by certain state will converge to the theoretical value.

Directions:

As the underlying model consider a loan quality model. Suppose a credit union classifies automobile

loans into four categories: Paid in Full (F), Good Standing (G), Behind Schedule (B), Collections

(C). Past records indicate that each month, accounts in good standing change as follows: a % pay the loan in full,b% fall behind on payments,(100ab)% remain in good standing. Similarly, bad loans historically change every month as follows:c % are paid in full,d% return to good standing,e% remain behind schedule,(100cde)% are sent to collection. Loans that are paid in full or sent to collection are the absorbing states. Suppose that right now, the bank has f% of B loans and (100f)% of G loans.To obtain your parameter values,

for a, b follow link https://www.wolframalpha.com/input/?i=two+random+numbers+between+1+and+50

for c, d, e follow link https://www.wolframalpha.com/input/?i=three+random+numbers+between+1+and+33

for f follow link https://www.wolframalpha.com/input/?i=one+random+number+between+1+and+100

You are tasked with finding the proportion of current loans that will be paid in full. To do that, first you need the probability that a loan that starts as G or B ends in either F or C, which we can denote as p_GF, p_GC, p_BF, p_BC.

Define transition matrix P. For example, in Python a 33 matrix can be defined by:

P = [ [ . 4 , . 4 , . 2 ] , [ . 0 , . 5 , . 5 ] , [ . 0 , . 4 , . 6 ] ]

Here the first row is (0.4,0.4,0.2), second row is(0,0.5,0.5), etc. In your case, you will need to define a 4 4 matrix with transition probabilities that you need.

Define an array of state counters (I will call it scnt) for the number of times you end up in each absorbing state; the number of simulation steps that you will use (I will call it nsteps); and the current state (state). Let us agree that the states of our Markov chain are ordered as (F, G, B, C), so state F corresponds to 0, state G corresponds to 1, etc. Initialize

n s t e p s = 1 0 0 0 0

Now, perform the following simulation procedure twice: first for a G loan, and then for a B loan.

1. Initialize

s t a t e = 1 ( o r 2 )

2. Perform transitions according to the transition matrix until you end in either of the absorbing

states. To do that repeat the following

perform a transition. The next state is determined by the current state (state) and the corresponding row of the transition matrix. There are different ways to do that, one of which is

to use a uniform random variable.

draw a realization of a uniform random variable (function u = random.random()) in Python

for the row of the transition matrix that corresponds to the current state, add the transition probabilities one by one from left to right until the sum exceeds the value of u. The

first time when that happens determines the next state

check if current state is F or C. If no, repeat step 2. If yes, record the result (increment the corresponding scnt).

3. Repeat steps 1 and 2 nsteps times to collect enough simulations

4. Compute the absorption proportions by dividing each scnt by nsteps.

The computational procedure above is outline in a pseudo-code at the end of this document.

Now that you have the probabilities, calculate the proportion of loans that will be repaid as f_pBF + (1f)p_GF, where p_BF and p_GF are the probabilities to be absorbed in state F starting from B or F that you calculated in the previous step.

Submission instructions

Submit your code and a brief (not more than one page) report on Canvas. In the report clearly identify the values of parameters that you used, the absorption proportions that you obtained

and the conclusions that you have made. The report should be in .pdf, .doc or .docx format. If you are using Python submit your .py file, otherwise, submit all of the files required to recreate your experiments.

2

Pseudo code for the assignment. First simulate the system starting from state G

n s t e p s = 1 0 0 0 0

s c n t = [ 0 , 0 , 0 , 0 ]

r e p e a t n s t e p s t i m e s :

s t a t e = 1

w h i l e ( s t a t e ==1 o r s t a t e = = 2 )

p e r f o r m a t r a n s i t i o n a c c o r d i n g t o t r a n s i t i o n m a t r i x P

s c n t [ s t a t e ] = s c n t [ s t a t e ] + 1

p_GF = s c n t [ 0 ] / n s t e p s

p_GC = s c n t [ 3 ] / n s t e p s

Now, do the same starting from state B

n s t e p s = 1 0 0 0 0

s c n t = [ 0 , 0 , 0 , 0 ]

r e p e a t n s t e p s t i m e s :

s t a t e = 2

w h i l e ( s t a t e ==1 o r s t a t e = = 2 )

p e r f o r m a t r a n s i t i o n a c c o r d i n g t o t r a n s i t i o n m a t r i x P

s c n t [ s t a t e ] = s c n t [ s t a t e ] + 1

p_BF = s c n t [ 0 ] / n s t e p s

p_BC = s c n t [ 3 ] / n s t e p s