2

What are the methods to solve a CT Markov process?

I know that for known processes such as birth-death or some queues, problem can be solved analytically? However, how to solve if it is not analytically solvable?

It looks that numerical method should be used. However, I have not found any tools that support it yet. MATLAB has ode method but in addition to setting up the (ordinary differential equation) ode of process, a linear equation should be added which matlab does not support?

How to solve these problems?
Any tools or any other method?

Yan Berk
  • 14,328
  • 9
  • 55
  • 52
justin
  • 99
  • 9

1 Answers1

0

General references

You may want to glance at the standard references that physical scientists use for continuous time Markov process. N.G. van Kampen's Stochastic Processes in Physics and Chemistry is a comprehensive and readable treatment. Gardiner's Stochastic Methods: A Handbook for the Natural and Social Sciences is another well-used reference. Both of these present both analytical solutions and perturbative methods for solving stochastic processes.

Discrete states

If your states are discrete (or you can approximate them discretely), then you can a Monte Carlo sampling methods to construct an ensemble of trajectories. One of the better references for this is Barkema and Newman's Monte Carlo Methods in Statistical Physics, which has nice examples of spin systems, and presents several algorithms. You will also find quite a bit of readable material in the literature on the Gillespie Algorithm and Kinetic Monte Carlo.

Continous states

If your process is formulated as a Langevin equation, or contains an similarly embedded Wiener process, then you'll probably want to look at techniques to integrate stochastic differential equations (SDEs). One reasonably recent paper with some good references Algorithms for Brownian dynamics computer simulations: Multivariable case by BraƄka and Heyes.

If your CT Markov process can be formulated as a Fokker-Planck equation, then Risken's The Fokker-Planck Equation: Methods of Solutions and Applications is an excellent place to look for analytical solutions. Fokker-Planck equations are very similar to diffusion-advection equations, are are similarly amenable to numerical PDE techniques.

Implementation

Most of these algorithms are reasonably easy to implement. Direct Monte Carlo methods are very simple to write. Continous-time Monte Carlo is only a bit more complex. Integrating SDEs is about as complex as writing a Runge-Kutte or a Verlet integrator, and often easier than numerically integrating a Fokker-Planck equation, but not always.

sfstewman
  • 5,589
  • 1
  • 19
  • 26