Stack Overflow
Stack Overflow

Questions tagged [markov-chains]

Markov chains are systems which transition from one state to another based only upon their current state. They are used widely in various statistical domains to generate sequences based upon probabilities.

Markov chains (named after their creator, Andrey Markov) are systems which transition from one state to another based only upon their current state. They are memoryless processes which are semi-random, i.e. where each state change having an associated probability.

Due to their statical nature, Markov chains are suitable for simulating complex real-life processes where probabilities are well known. They are used in a wide variety of fields, with uses too in-depth to list here; an exhaustive list can be found on the associated Wikipedia page.

In programming, they are especially popular for manipulating human languages - Markov text generators are especially popular applications of Markov chains.

577 questions
12
votes
1 answer

Simulating a Markov Chain with Neo4J

A Markov chain is composed of a set of states which can transition to other states with a certain probability. A Markov chain can be easily represented in Neo4J by creating a node for each state, a relationship for each transition, and then…
JnBrymn
  • 24,245
  • 28
  • 105
  • 147
11
votes
2 answers

iPython Notebook; Plotting transition diagrams

My question is dead simple. Is there a package to plot state-transition or markov diagrams that look like any of the following? I am thinking it has to exist, but I simply can't find it! I've really had a search around, also on Stackoverflow, but…
tmo
  • 1,393
  • 1
  • 17
  • 47
11
votes
1 answer

What is the best/fastest way to construct a very large markov chain from simulation data?

I have written a C++ program that simulates a certain process I'm studying. It outputs discrete "states" each timestep of the simulation. For example: a b c b c b would be the output of a simulation run with a as the initial condition (set by me or…
jlmr
  • 165
  • 10
10
votes
3 answers

Optimisation of recursive algorithm in Java

Background I have an ordered set of data points stored as a TreeSet. Each data point has a position and a Set of Event objects (HashSet). There are 4 possible Event objects A, B, C, and D. Every DataPoint has 2 of these, e.g. A and…
bountiful
  • 814
  • 1
  • 8
  • 22
9
votes
1 answer

Efficient implementation of Markov Chains in julia

I want to simulate the movement of a random walker in a network as efficiently as possible. Below I show a toy model with the three approaches I have tried so far. I should note that in my original problem the edges of the network are fixed, however…
RM-
  • 986
  • 1
  • 13
  • 30
9
votes
2 answers

How to test if a string contains gibberish?

I am making a registering form and because some will enter gibberish in the Secret Answer's input (I do that myself), I would like to test that value to see if it's likely to be a good answer. I have a function that generates Markov Chains (at…
Cybrix
  • 3,248
  • 5
  • 42
  • 61
9
votes
1 answer

Why should we use RNNs instead of Markov models?

Recently I stumbled across this article, and I was wondering what the difference between the results you would get from a recurrent neural net, like the ones described above, and a simple Markov chain would be. I don't really understand the linear…
Justin Sanders
  • 313
  • 2
  • 12
9
votes
1 answer

How to vectorize a random walk simulation in MATLAB

I am rewriting a Monte Carlo simulation model in MATLAB with an emphasis on readability. The model involves many particles, represented as (x,y,z), following a random walk over a small set of states with certain termination probabilities. The…
turtlesoupy
  • 325
  • 4
  • 7
9
votes
2 answers

What are the differences between Monte Carlo and Markov chains techniques?

I want to develop RISK board game, which will include an AI for computer players. Moreovor, I read two articles, this and this, about it, and I realised that I must learn about Monte Carlo simulation and Markov chains techniques. And I thought that…
quartaela
  • 2,579
  • 16
  • 63
  • 99
8
votes
3 answers

Graphical markov chain in javascript

I have a Markov chain that I would like to represent graphically in javascript. I need to represent the nodes, links, and transition probabilities. Perhaps something like one of these two diagrams: Finding a good image library (like Raphael) is…
Jeff
  • 12,147
  • 10
  • 51
  • 87
8
votes
3 answers

Finding stationary distribution of a markov process given a transition probability matrix

There has been two threads related to this issue on Stack Overflow: How can I obtain stationary distribution of a Markov Chain given a transition probability matrix describes what a transition probability matrix is, and demonstrate how a…
Zheyuan Li
  • 71,365
  • 17
  • 180
  • 248
8
votes
1 answer

Change the size of the arrowheads in a markov chain plot

I've plotted a markov chain in R, but I dislike the rather hugh arrowheads that the plot-function is plotting. Is there a way to make the heads smaller? library( markovchain ) transition.matrix <- matrix( data = c( 0.5, 0, 0, 0.5, 0.2, 0, 0, 0.8, 1…
JimBoy
  • 597
  • 8
  • 18
8
votes
1 answer

Constructing a multi-order Markov chain transition matrix in Matlab

A first-order transition matrix of 6 states can be constructed very elegantly as follows x = [1 6 1 6 4 4 4 3 1 2 2 3 4 5 4 5 2 6 2 6 2 6]; % the Markov chain tm = full(sparse(x(1:end-1),x(2:end),1)) % the transition matrix. So here is my…
pangyuteng
  • 1,749
  • 14
  • 29
7
votes
3 answers

Algorithm for computing the plausibility of a function / Monte Carlo Method

I am writing a program that attempts to duplicate the algorithm discussed at the beginning of this article, http://www-stat.stanford.edu/~cgates/PERSI/papers/MCMCRev.pdf F is a function from char to char. Assume that Pl(f) is a 'plausibility'…
Sean Thoman
  • 7,429
  • 6
  • 56
  • 103
7
votes
1 answer

Reconstructing now-famous 17-year-old's Markov-chain-based information-retrieval algorithm "Apodora"

While we were all twiddling our thumbs, a 17-year-old Canadian boy has apparently found an information retrieval algorithm that: a) performs with twice the precision of the current, and widely-used vector space model b) is 'fairly accurate' at…
silverasm
  • 501
  • 5
  • 10
1
2
3
…
38 39