2

Given a markov model, which has a start state named S and an exit state named F, and this model can be represented as a directed graph, with some constraints:

  1. Every edge has some weight falls in the range (0,1] as the transition probability.

  2. Weights of the edges coming out from each node sum to 1.

example markov model for this question

The question is how to rank the paths between start state and exit state? Or, more precisely, how to find out the path with highest probability?

On one hand, the weights are probabilities, so the longer the path, the smaller would the products be, so one heuristic strategy is to pick shorter path and bigger weight candidates; but can this problem be converted into shortest path problem or using some tailored Viterbi algorithm or some DP algorithm to solve?

justhalf
  • 8,960
  • 3
  • 47
  • 74
caravan eat
  • 21
  • 2

1 Answers1

9

Convert your probabilities to log space (the log base doesn't matter). Now the probability of a path becomes the sum of the log space weights (because log(ab) = log(a) + log(b) . Since the weights/probabilities are <1 the weights in log space will all be negative and the path has the highest weight.

To bring it more into the regular problem you can negate all the log space weights so that they are all positive and you are looking for the lowest sum. At this point you can run standard algorithms (Dijkstra would be simple and very fast) to find the path you are looking for. If you have the sum then negate it and calculate exponential to get the probability.

TL;DR: replace all weight w with -log(w) and run Dijkstra with the new weights.

Sorin
  • 11,863
  • 22
  • 26
  • thanks Sorin I just realized that my bad math mess myself. yes, -log(w) will satisfy the shorter path and larger probability, the 'shorter' path it will output. That's a question seems so stupid now. – caravan eat Aug 11 '16 at 12:15
  • Usually in Markov model you use Viterbi instead of Dijkstra, though – justhalf Aug 11 '16 at 12:18
  • one more question, will djikstra apply in this situation that it will probably has loops in the graph? @Sorin – caravan eat Aug 11 '16 at 12:19
  • Viterbi without modification? I think Viterbi is for HMM, if there is some tailored version for markov model, I'd Like to know! @justhalf – caravan eat Aug 11 '16 at 12:21
  • DAG means the nodes have topological order. You just traverse each node in topological order, taking max from all parent. I just saw your diagram now. Your diagram in the question is not DAG, it has cycle A-B-C-D-A. DAG means Directed **acyclic** graph. If it's not DAG you can't use my method. But Dijkstra should be fine. – justhalf Aug 11 '16 at 13:11
  • @caravaneat Yes, dijkstra will work for any graph (including loops), as long as all the weights are positive (assuming you do `-log(w)`). – Sorin Aug 11 '16 at 14:01