Questions tagged [markov]

Markov, or markov property refers to the memoryless property of a stochastic process.

Overview

From Wikipedia,

A stochastic process has the Markov property if the conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it.

A Markov process (Yt) can be expressed like below:

enter image description here

Tag usage

Please consider stack-exchange's Cross Validated SE for asking questions concerning statistics data analysis.

255 questions
1
vote
1 answer

MFCC Vector Quantization for Speaker Verification Hidden Markov Models

I am currently doing a project on speaker verification using Hidden Markov Models. I chose MFCC for my feature extraction. I also intend to apply VQ to it. I have implemented HMM and tested it on Eisner's data spreadsheet found here:…
Bobby
  • 31
  • 5
1
vote
1 answer

Markov chain simulation

I'm wondering if there is an algorithm to simulate a discrete Markov chain with a specific number of occurrences of state knowing the transition matrix way. For example, how to simulate in R a Markov chain of length n with p occurrences (p < n) of…
Armel
  • 31
  • 1
  • 5
1
vote
0 answers

MRF c++ source code

Can anyone suggest me a good c++ library for GMM based MRF learning and inference using Belief propagation? If this is not the right stack exchange site to ask this question, please suggest me the right one..
Varo
  • 831
  • 1
  • 7
  • 16
1
vote
0 answers

Does this class of MDP have an efficient solution?

I've been working on making a game solver for about a month now, trying a variety of strategies but most have been oriented around brute force. This works for simpler cases of the game but it fails for more complex cases (higher game tree depth).…
1
vote
2 answers

Using HIdden Markov Model for prediction

Suppose there is a sequence of observations,e.g. [1,2,3,5,5,5,2,3,2,3, ..., 3, 4]. I am trying to use the current implementation of HMM in Scikit-learn to predict the next value of this observation sequence. I have 2 questions regarding this. Given…
1
vote
1 answer

Degree of Freedom of Markov Chains

I have a set of 5000 strings of length 4, where each character in the string can be either A, B, C, or D. 0-order Markov Chain (no dependency), makes a 4*1 array of columns A, B, C, D. 1-order Markov Chain (pos j depends on previous pos i), makes a…
user1830307
1
vote
1 answer

Generating random numbers for solar energy harvesting using Markov models

How do I generate random numbers using a Markov model in C#? I noticed here that almost all of the applications of the Markov algorithm is for randomly writing text. Is there a source code somewhere or a tutorial where I can fully understand how…
Nico Encarnacion
  • 87
  • 1
  • 1
  • 8
1
vote
0 answers

Markov Decision Process Clarification

I'm in the process of implementing Value Iteration for a homework assignment. It's coming along nicely but I'm confused about a certain part, specifically the line indicated below //(taken from http://aima.cs.berkeley.edu/python/mdp.html) def…
Daniel
  • 2,435
  • 5
  • 26
  • 40
1
vote
2 answers

Converting Pseudocode to C++

I am trying to learn about Markov decision problems and I was given the algorithm for Value Iteration, but I am confused how to turn them into actual C++ code. Mainly the parts where summations and such occur. Here is the algorithm: function…
James Brown
  • 919
  • 3
  • 13
  • 22
1
vote
1 answer

Markov Algorithm for Random Writing

I got a litte problem understanding conceptually the structure of a random writing program (that takes input in form of a text file) and uses the Markov algorithm to create a somewhat sensible output. So the data structure i am using is to use…
Tom Lilletveit
  • 1,872
  • 3
  • 31
  • 57
1
vote
1 answer

"programming pearls": Strings of Pearls

In column 15.3, the author introduced how to generate text randomly from an input document. The author also gave the source code. qsort(word, nword, sizeof(word[0]), sortcmp); int sortcmp(char **p, char **q) { return wordncmp(*p, *q); } I've…
Fihop
  • 3,127
  • 9
  • 42
  • 65
0
votes
1 answer

Which is faster? Casting and subtractions or dict look ups in Python

So for fun, I decided to revisit an old college assignment I had in which a ciphertext was given of about 75 characters, and a crib that the message was signed with three letters (initials of my teacher) What I've done: Hemmed down the results to…
0
votes
0 answers

Single state Markov Model

I am trying to understand how Hidden Markov Models work. Using hmmlearn library, I want to see if a time-series can be posible under a learned distribution. I start with a single state markov model. I generate a gaussian sequence with mean 0 and…
0
votes
0 answers

Adjusting for Hierarchical Clustering in Markov Model

I'm interested in using a discrete-time non-homogeneous Markov Model for one of my studies. The data I am using has Hierarchical clustering (ex. patients within hospitals). I was wondering if anyone knows how to account for clustering in such a…
0
votes
0 answers

find CDF of a markov fluid process having PDF

I have a markov process with 5 states and I have calculated its probability density function using this code % Define the function f = @(y) a0*L_0 + a_neg*expm(A_neg * y)*L_neg + a_pos*exp(-A_pos * (2-y))*L_pos; pointnums=1000; y_values =…
nemo92world
  • 101
  • 8