Question about constructing transition matrix for a scenario

Question

I am facing some problem on constructing transition probability matrix when I am studying and following is the scenario of the question:

**Assuming a phone has had i faults (for i = 0,1,2,3 the probability of having another fault is p, independently of all previous day, and the new fault is to be repaired with probability q^i+1 within the same day.

If a phone is not repaired, it is replace instead.

If a phone has its n'th fault, it is immediately replaced. (n=3)

The replacement phone initially havve zero faults

The probability of experiencing more than one fault within single day is sufficiently small and is to be ignored.**

How do I construct a transition matrix for a Markov Chain whose current state represents the number of faults a phone has experienced (and had repaired) up to the current day.

Answer 1

You can start with an easier case: assume the phone can be repaired only on the next day. Let's define all the possible states:

• 0f OK - zero faults and the phone is OK
• 1f B - one fault and the phone is broken
• 1f OK - one fault and the phone is OK
• 2f B - two faults and the phone is broken
• 2f OK - two faults and the phone is OK
• 3f B - three faults and the phone is broken

Now you can draw a graph for this case:

Now we know the phone actually must be repaired on the same day. In this case we have only 3 states and in each state the phone is OK (either repaired or replaced):

• 0f - zero faults
• 1f - one fault
• 2f - two faults

From the first graph you can see all the paths which go from one OK state to another OK state. All what you need to do is to follow the paths and collect the probabilities:

The transition matrix would look like this:

You can see that each raw sums up to 1 (there are two different conventions. the second one says the columns should sum up to 1). In the first convention you calculate the next state vector like X=A*X and in the second one like X=X*A