Questions tagged [markov-chains]

Markov chains are systems which transition from one state to another based only upon their current state. They are used widely in various statistical domains to generate sequences based upon probabilities.

Markov chains (named after their creator, Andrey Markov) are systems which transition from one state to another based only upon their current state. They are memoryless processes which are semi-random, i.e. where each state change having an associated probability.

Due to their statical nature, Markov chains are suitable for simulating complex real-life processes where probabilities are well known. They are used in a wide variety of fields, with uses too in-depth to list here; an exhaustive list can be found on the associated Wikipedia page.

In programming, they are especially popular for manipulating human languages - Markov text generators are especially popular applications of Markov chains.

577 questions
0
votes
1 answer

fitting transition matrix to crosstable in r

I am having some trouble fitting a transtion matrix. While analysing panel data I need to fit a transition matrix (with standard error) to the crosstable shown below or the underlying data. It shows participants transitioning between income states…
0
votes
1 answer

missing value where TRUE/FALSE needed in Markov-Chain

I have been trying to use Markov Chain to improve my model and get trouble when computing transition matrix. It appears missing values. Someone know why my code is wrong? Many thanks I already defined all the variables to be 0 at first. mresiduals…
June Choo
  • 13
  • 3
0
votes
1 answer

Transition time between states

I have a rather simple question. I have system with different states, departure rate = mu and arrival rate = lambda. If i want to calculate the time to leave a state i want to do the following: time_to_transition <- .... Obviously i need help with…
PeterNiklas
  • 75
  • 1
  • 9
0
votes
2 answers

Markov chain. Implementation add method

I am a bit confused with the next statment in implementation: void add(char *prefix[NPREF], char *suffix) { State *sp; sp = lookup(prefix, 1); //addsuffix(sp, suffix); printf("size of prefix %d",&prefix); printf("size of prefix…
0
votes
2 answers

Markov Chain previous states

I've simulated a 1000 steps in a markov chain were there are in total 6 different states(0-5) and we started in state 5. With the bar plot we can we see how many times we are in each state. However, what i want to know is how many times we went to…
PeterNiklas
  • 75
  • 1
  • 9
0
votes
3 answers

How to calculate "compound" Markov transition matrix in Stata or R?

By "compound" I mean the transition matrix satisfies the Markov property,namely I have two columns s_t and s_t+k that represent state of each individual in two period t and t+k respectively. What I want is to find the matrix M that s_t+k = M^k *…
zlqs1985
  • 509
  • 2
  • 8
  • 25
0
votes
2 answers

Identifying absorbing classes in a sparse graph

My problem is as follows. I have Markov chain with several absorbing classes and I would like to determine which states in the state space are absorbing and which are transient. As a simple example, we could have the following sparse transition…
Forzaa
  • 1,465
  • 4
  • 15
  • 27
0
votes
0 answers

Huge deegree of markov chain matrix

we met with interesting problem of R. We wanted to find 100 degree of any given markov chain matrix. The problem is that after some time matrix suddenly goes to zero. We think that is causation of approximately of matrix multiplication. Do you…
0
votes
3 answers

Writing to a File, Adding an Additional Space

I am creating a Markov text generator that generates haikus. The function to generate the haiku itself will generate 100 haikus using a for loop. They may look something like: line1 line2 line3 line1 line2 line3 line1 line2 line3 When I try writing…
natalien
  • 81
  • 3
  • 13
0
votes
3 answers

how to programme in Matlab to obtain transition probability matrix?

I have a sequence x= [12,14,6,15,15,15,15,6,8,8,18,18,14,14] so I want to make transition probability matrix. Transition probability matrix calculated by equation i.e. probability=(number of pairs x(t) followed by x(t+1))/(number of pairs x(t)…
Ram s
  • 21
  • 5
0
votes
1 answer

Continuous-time finite-horizon MDP

Is there any algorithm for solving a finite-horizon semi-Markov-Decision-Process? I want to find the optimal policy for a sequential decision problem with a finite action space, a finite state space, and a deadline. Critically, different actions…
0
votes
1 answer

Calculating probability to get from s1:s400 to sn with large Markov transition matrix in R takes forever

I successfully implemented the following task, but I only got it to work with a small test dataset. For my real dataset, it just keeps calculating forever. I have a 400x400 transition probability matrix in R. A user hits "Conversion" if she converts…
0
votes
1 answer

Fine-tuning Markov chain

I have the following Markov…
Oposum
  • 1,155
  • 3
  • 22
  • 38
0
votes
2 answers

Proportions in Markov chain do not add up to 1

I have following two-state Markov…
Oposum
  • 1,155
  • 3
  • 22
  • 38
0
votes
1 answer

Unable to map a matrix in Matlab

I have this 3x3 system of equations for 2-D markov chain. I want to map the coefficients of this matrix to look like this 9x9 matrix having those zeros coefficients as well but i don't know how to proceed with it. Any help ? [ 5*P11 - 2*P12 - 2*P21,…
A Ahmed
  • 7
  • 6