Stack Overflow
Stack Overflow

Questions tagged [markov-chains]

Markov chains are systems which transition from one state to another based only upon their current state. They are used widely in various statistical domains to generate sequences based upon probabilities.

Markov chains (named after their creator, Andrey Markov) are systems which transition from one state to another based only upon their current state. They are memoryless processes which are semi-random, i.e. where each state change having an associated probability.

Due to their statical nature, Markov chains are suitable for simulating complex real-life processes where probabilities are well known. They are used in a wide variety of fields, with uses too in-depth to list here; an exhaustive list can be found on the associated Wikipedia page.

In programming, they are especially popular for manipulating human languages - Markov text generators are especially popular applications of Markov chains.

577 questions
0
votes
1 answer

Markov chain fit from many individual chains (in R)

Using the markovchain package, I'm working with a dataset comprised of six monthly observations for each of 23k individuals. When I go to fit a DTMC using the markovchainFit function, the function appears to want to take in what would be just one…
Andrew Cheesman
  • 140
  • 1
  • 10
0
votes
3 answers

Error in parallelization MPI_Allgather

EDITED IN LIGHT OF THE COMMENTS I am learning MPI and I am doing some exercises to understand some aspects of it. I have written a code that should perform a simple Monte-Carlo. There are two main loops in it that have to be accomplished: one on the…
Fra
  • 161
  • 8
0
votes
1 answer

Moving players on chance on a soccer field in R plot

Background: I'm trying to come up with a coding scheme such that any of my 14 players (separately; each player is shown as numbered circles below) on the center thick line of my soccer field independently moves to left by .1 (in x-axis value unit)…
rnorouzian
  • 7,397
  • 5
  • 27
  • 72
0
votes
0 answers

Lua: Markov-chain algorithm error

I am trying to run my code, but when I do it, it gives me the error: input:1: attempt to index a nil value (global 'arg'). I do not know enough to troubleshoot it. Here is my code: function allwords() local line = io.read() -- current…
Josh
  • 71
  • 1
  • 2
  • 11
0
votes
1 answer

What is the significance of the stationary distribution of a markov chain given it's initial state?

Let X_n be a MC, P not regular Say we have a stationary dist (pi_0, ..., pi_n) and P(X_0 = i) = 0.2, does this say anything? To be more clear: I ask because Karlin says when a stationary dist is not a limiting dist, P(X_n = i) is dependent on the…
user6437583
  • 75
  • 1
  • 1
  • 10
0
votes
2 answers

Reading into hashmap in c++

I'm working on a markov chain and have created a 2d hashmap that calculates the weighted probabilities. The output of this works fine. I'm looking to find the best way to output the next value. The way I have it at the moment isn't working…
Joshua Hodge
  • 17
  • 5
0
votes
1 answer

MarkovChain package error

I have a problem with making a Markov model using the library 'markovchain'. To make a markovchain plot you need to have a transistion matrix. As you see the rows sums up to 1. > rowSums(trans_matrix) (start) (conversion) …
PeterW
  • 27
  • 1
  • 7
0
votes
2 answers

Gibbs sampling gives small probabilities

As part of our final design project, we have to design a Gibbs sampler to denoise an image. We have chosen to use the Metropolis Algorithm instead of a regular Gibbs sampler. A rough sketch of the algorithm is as follows, all pixels are 0-255…
Richard Hum
  • 651
  • 2
  • 6
  • 14
0
votes
1 answer

Programming a discord bot in python, ran into the error "string index out of range"

I'm making a discord bot that spurts out randomly generated sentences into the chat every few seconds. Im trying to use the nltk module to make the sentences more coherent, but I'm caught up on an error and cant figure it out. import asyncio import…
Museman
  • 7
  • 1
  • 6
0
votes
1 answer

Making a discord bot, but keep getting "missing 1 required positional argument: 'self'"

I'm making a discord bot that spurts out randomly generated sentences into the chat every few seconds. Im trying to use the nltk module to make the sentence structure a little better, but I'm caught up on an error and cant figure it out.(I'm newish…
Museman
  • 7
  • 1
  • 6
0
votes
0 answers

Monte Carlo Simulation with chaning distribution

I am experimenting with Monte Carlo Simulations and I have come up with this interesting problem. Suppose we are generating random values using a Normal distribution with St.Dev = 2 and mean = the last value generated (Markov process), we start at…
JJMenga
  • 1
0
votes
1 answer

Hidden Markov Model Bayesian Relation

Hi all this is artificial intelligence class from udacity. I have a question. P(R0)=1 means probability of day0 rainy is is 1. Here is my question P(R2 | H1 G2)? meaning we know I am happy at day1 and grumpy at day2 what is probability it is…
trial
  • 19
  • 1
  • 6
0
votes
0 answers

Markov Process (Using R programming)

Consider Markov Process Xt on 5 states{1,2,3,4,5} with associated rate matrix Q <- matrix( c(-3,3,0,0,0,0,-2,2,0,0,0,1,-1,0,0,0,0,0,-3,3,1,0,2,0,-3), nrow=5, ncol=5, byrow = TRUE) How to produce R code to estimate the probability that this process…
John Tan
  • 1
  • 1
0
votes
1 answer

Given transition matrix for Markov chain of 5 states, find first passage time and recurrence time

Transition matrix for a Markov chain: 0.5 0.3 0.0 0.0 0.2 0.0 0.5 0.0 0.0 0.5 0.0 0.4 0.4 0.2 0.0 0.3 0.0 0.2 0.0 0.5 0.5 0.2 0.0 0.0 0.3 This is a transition matrix with states {1,2,3,4,5}. States {1,2,5} are recurrent and…
June
  • 15
  • 5
0
votes
1 answer

Good *free* markov modeling tools?

I would like to use Markov models for some architecture simulations, but don't have a budget to buy anything like, eg, SHARPE. Does anyone know of a freeware tool, either platform-independent or available for Mac OS/X
Charlie Martin
  • 110,348
  • 25
  • 193
  • 263
1 2 3
…
38 39