Stack Overflow
Stack Overflow

Questions tagged [markov]

Markov, or markov property refers to the memoryless property of a stochastic process.

Overview

From Wikipedia,

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it.

A Markov process (Yt) can be expressed like below:

enter image description here

Tag usage

Please consider stack-exchange's Cross Validated SE for asking questions concerning statistics data analysis.

255 questions
0
votes
1 answer

Python statsmodel.tsa.MarkovAutoregression using current real GNP/GDP data

I have been using statsmodel.tsa.MarkovAutoregressio to replicate Hamilton's markov switching model published in 1989. If using the Hamilton data (real GNP in 1982 dollar) I could have the same result as the code example / the paper showed. However,…
INNS
  • 1
  • 2
0
votes
0 answers

Monte Carlo Simulation with chaning distribution

I am experimenting with Monte Carlo Simulations and I have come up with this interesting problem. Suppose we are generating random values using a Normal distribution with St.Dev = 2 and mean = the last value generated (Markov process), we start at…
JJMenga
  • 1
0
votes
2 answers

How to print MSM R package results in Rmarkdown MS Word output?

I am using Rmarkdown and Kniter in Rstudio to make MS Word document. However, my results are from MSM package. This is my code in the .rmd sript: knitr::kable(pmatrix.msm(xdata.msm),caption = "Transition probability matrix.") I am getting this…
runjumpfly
  • 319
  • 1
  • 10
0
votes
0 answers

transition probabilty of being in bottom row is 1/6

I'm currently studying transition probabilities using Markov chain diagrams which I thought I understood. However I'm now required to find the probabilities of being in the bottom row of a 3x3 grid is 1/6 and the probability of being in the middle…
cazorlaYo
  • 11
  • 1
0
votes
1 answer

Transform Data format for "channelAttribution" package in R

I have the following data format - Id Record Date Medium 1 display 9/7/2016 A 1 display 9/8/2016 B 1 display 9/9/2016 A 1 Interaction 9/10/2016 B 1 display 9/11/2016 A 1 …
Rohit Lodha
  • 43
  • 8
0
votes
1 answer

Transition time between states

I have a rather simple question. I have system with different states, departure rate = mu and arrival rate = lambda. If i want to calculate the time to leave a state i want to do the following: time_to_transition <- .... Obviously i need help with…
PeterNiklas
  • 75
  • 1
  • 9
0
votes
2 answers

Markov Chain previous states

I've simulated a 1000 steps in a markov chain were there are in total 6 different states(0-5) and we started in state 5. With the bar plot we can we see how many times we are in each state. However, what i want to know is how many times we went to…
PeterNiklas
  • 75
  • 1
  • 9
0
votes
3 answers

How to calculate "compound" Markov transition matrix in Stata or R?

By "compound" I mean the transition matrix satisfies the Markov property，namely I have two columns s_t and s_t+k that represent state of each individual in two period t and t+k respectively. What I want is to find the matrix M that s_t+k = M^k *…
zlqs1985
  • 509
  • 2
  • 8
  • 25
0
votes
1 answer

Why is the environment state markov?

I had a question related to reinforcement learning, why is the environment state markov? I read it somewhere it is by definition but I can't understand how the definition of environment state implies that is markov.
Abhishek Bhatia
  • 9,404
  • 26
  • 87
  • 142
0
votes
0 answers

OCR using a hidden Markov model

I want to recognise English alphabets using a hidden Markov model. I have extracted features using the zoning method. I want to use HTK toolkit for the training. What is the format for giving the feature matrix as input to HTK? How should I give…
Adithya Kasarla
  • 1
  • 1
0
votes
4 answers

Is there a resource of lots of human text?

I just coded a Markov chain that talks based on learned data. I'd like a resource of a lot of text data online, but can't seem to find any (most sites like Wikipedia have a lot of junk, not plain text files). Is there any site that would have a lot…
Cisplatin
  • 2,860
  • 3
  • 36
  • 56
0
votes
1 answer

Error in Markov-switching VAR in R

I'm trying to estimate a Markov-switching VAR in R using the command msvar. These are the first 10 entries of my two time series. I have 798. When I try to run this I get an Error message a <- c(1.998513, 1.995302, 2.030693, 2.122130, 2.236770,…
Enrico Forabosco
  • 33
  • 1
  • 7
0
votes
1 answer

Creating a transition matrix in R using sea turtle data

This has been rattling around my head for a while. From Broderick et al. 2006 (http://www.ascension-island.gov.ac/wp-content/uploads/2012/12/are-green-turtles-globally-endangered.pdf), they created a transition matrix (Table 1) for the probability…
Constantin
  • 132
  • 9
0
votes
1 answer

If condition doesn't trigger

Long story short story, I am playing with the tweepy library to make a markov bot on twitter: I am now trying to answer to the right tweet but... >>> repr(mention.id) '698147667990441984' …
Nemecle
  • 1
0
votes
1 answer

Two problems on writing a script to compute markov joint distribution (in python)

I'm a new-learner of python, recently I'm working on some project to perform computation of Joint distribution of a markov process. An example of a stochastic kernel is the one used in a recent study by Hamilton (2005), who investigates a nonlinear…
zlqs1985
  • 509
  • 2
  • 8
  • 25
1 2 3
…
16 17