Markov Transition Probability Matrix Implementation in Python

Question

I am trying to calculate one-step, two-step transition probability matrices for a sequence as shown below :

sample = [1,1,2,2,1,3,2,1,2,3,1,2,3,1,2,3,1,2,1,2]
import numpy as np

def onestep_transition_matrix(transitions):
    n = 3 #number of states

    M = [[0]*n for _ in range(n)]

    for (i,j) in zip(transitions,transitions[1:]):
        M[i-1][j-1] += 1

    #now convert to probabilities:
    for row in M:
        s = sum(row)
        if s > 0:
            row[:] = [f/s for f in row]
    return M

one_step_array = np.array(onestep_transition_matrix(sample))

My question, how do we calculate two step transition matrix. because when I manually calculate the matrix it is as below :

two_step_array = array([[1/7,3/7,3/7],
                       [4/7,2/7,1/7],
                       [1/4,3/4,0]])

However. np.dot(one_step_array,one_step_arrary) gives me a result which is different and as follows :

array([[0.43080357, 0.23214286, 0.33705357],
   [0.43622449, 0.44897959, 0.11479592],
   [0.20089286, 0.59821429, 0.20089286]])

Please let me know which one is correct.

score 5 · Accepted Answer · answered Sep 03 '18 at 05:59

5

You just have to change the transitions index in your for loop:

def twostep_transition_matrix(transitions):
    n = 3 #number of states

    M = [[0]*n for _ in range(n)]

    for (i,j) in zip(transitions,transitions[2:]):
        M[i-1][j-1] += 1

    #now convert to probabilities:
    for row in M:
        s = sum(row)
        if s > 0:
            row[:] = [f/s for f in row]
    return M

answered Sep 03 '18 at 05:59

Ernest S Kirubakaran

1,524
12
16

Which means two-step transistion matrix is not same as matrix Power 2. Is my understanding correct? – Nadipineni Naimisha Sep 03 '18 at 06:04
2

Yes, both are not the same – Ernest S Kirubakaran Sep 03 '18 at 06:06
2

The difference is the above is the actual two-step transfer matrix, while the power is the *estimate* of the two-step transfer matrix based on the one-step transfer matrix. With such a small sample size the estimate and the reality are not likely to be the same, even if your Markov process is memoryless. – Daniel F Sep 03 '18 at 10:06
@ErnestSKirubakaran, what is the reason for this line `row[:] = [f/s for f in row]`? Why can't we assign directly to `row`? Sorry, I'm newbie in python. – Mark Nov 27 '19 at 17:18
1

@Mark In the for loop, `row` is the object that is referring to the particular row in the matrix M. If we give row[:] = [f/s for f in row], the reference of the row will change from the corresponding row in M to the new row `[f/s for f in row]`. But the original row in M will not be changed. If we want to change the row in M, we need to give `row[:] = [f/s for f in row]` – Ernest S Kirubakaran Nov 29 '19 at 04:28
@ErnestSKirubakaran, thanks for your comment. So, since `row` in the for loop is a reference, changing it will not change what it points to? But how does `row[:]` make it not a reference? – Mark Nov 30 '19 at 03:22
`row[:]` still refers to the original `row` and doesn't create another object – Ernest S Kirubakaran Dec 03 '19 at 09:50
@ErnestSKirubakaran, one more question. In the code you provided, when the probabilities are calculated, are they also normalized per row? – Mark Dec 05 '19 at 03:05
@Mark If by normalization per row you mean that the rows should add upto 1, then yes, the probabilities are normalized per row – Ernest S Kirubakaran Dec 05 '19 at 09:53

Markov Transition Probability Matrix Implementation in Python

1 Answers1