Questions tagged [stochastic-process]

A stochastic process is a collection of related random variables, often used as a model for a quantity that varies over time or space with some degree of smoothness.

In probability theory, a stochastic process, or sometimes random process (widely used) is a collection of random variables; this is often used to represent the evolution of some random value, or system, over time. This is the probabilistic counterpart to a deterministic process (or deterministic system). Instead of describing a process which can only evolve in one way (as in the case, for example, of solutions of an ordinary differential equation), in a stochastic or random process there is some indeterminacy: even if the initial condition (or starting point) is known, there are several (often infinitely many) directions in which the process may evolve.

In the simple case of discrete time, a stochastic process amounts to a sequence of random variables known as a time series (for example, see Markov chain). Another basic type of a stochastic process is a random field, whose domain is a region of space, in other words, a random function whose arguments are drawn from a range of continuously changing values. One approach to stochastic processes treats them as functions of one or several deterministic arguments (inputs, in most cases regarded as time) whose values (outputs) are random variables: non-deterministic (single) quantities which have certain probability distributions. Random variables corresponding to various times (or points, in the case of random fields) may be completely different. The main requirement is that these different random quantities all have the same type. Type refers to the co-domain of the function. Although the random values of a stochastic process at different times may be independent random variables, in most commonly considered situations they exhibit complicated statistical correlations.

Familiar examples of processes modeled as stochastic time series include stock market and exchange rate fluctuations, signals such as speech, audio and video, medical data such as a patient's EKG, EEG, blood pressure or temperature, and random movement such as Brownian motion or random walks. Examples of random fields include static images, random terrain (landscapes), wind waves or composition variations of a heterogeneous material.

143 questions
1
vote
2 answers

Simulate a Markov chain path 1000 times

I have the following code for a Markov chain: simulation.mc=function(i0,P,n.sim){ S=1:nrow(P) X=rep(0,n.sim) X[1]=i0 for (n in 2:n.sim){ X[n]=sample(x=S,size=1,prob=P[X[n-1],]) } return(X) } P=matrix( c( 0,1/2,0,1/2,0,0,0, …
Maruska
  • 13
  • 4
1
vote
1 answer

Is there Implementation of Hawkes Process in PyMC?

I want to use Hawkes process to model some data. I could not find whether PyMC supports Hawkes process. More specifically I want an observed variable with Hawkes Process and learn a posterior on its params. If it is not there, then could I define it…
Harit Vishwakarma
  • 2,338
  • 4
  • 21
  • 36
1
vote
1 answer

Bayesian Stochastic Optimal Control, MCMC

I have a Stochastic Optimal Control problem that I wish to solve, using some type of Bayesian Simulation based framework. My problem has the following general structure: s_t+1 = r*s_t(1 - s_t) - x_t+1 + epsilon_t+1 x_t+1 ~ Beta(u_t+1, w_t+1) u_t+1 =…
P_J
  • 67
  • 9
1
vote
0 answers

Range of Autoregressive Coefficients

I was wondering if there is a "rule of thumb" range (or something more concretely) for AR coefficients that would indicate whether or not a process is strongly mean reverting, mean reverting, random walk and etc. Any advice would be greatly…
user5087936
1
vote
0 answers

Finding stationary distribution of a huge Markov Chain

(Updated) I need to find stationary distribution of a huge Markov Chain. My state vector is (t,x_1,x_2,x_3,x_4,x_5) where t=1,...,7 and each of x_j=0,...,40. As you can see, the size of transition matrix is huge. I'm trying to solve P=PxA where P…
NNsr
  • 1,023
  • 2
  • 11
  • 25
1
vote
1 answer

Markov Chain Monte Carlo Simulation Prooblem

I'm trying to run a MC simulator for a Markov Chain that is uniformly distributed among all NxN matrices that have no neighboring 1's. My algo is supposed to fill up the state space by running the chain a bunch of times. However there's something…
EhBabay
  • 149
  • 8
1
vote
2 answers

binary matrix of all combination with equal probability of 1 and 0 in matlab

I want to generate a binary matrix, let say (8,1). with equal probability means four 1's and four 0s in a matrix. with different permutation of these elements total 70 combination are possible (e.g 8C4). I want these all possible combination one by…
1
vote
1 answer

Random bits array by given probabilities with numpy

I have a deterministic neural network and I want to make it stochastic. Two questions: I'm not sure if it means that I need to use the result of the sigmoid to determine the probabilities for the output, or if the probabilities are simply the…
Uri
  • 25,622
  • 10
  • 45
  • 72
1
vote
1 answer

CIR model estimation through MLE

I want to estimate CIR model parameters though ML in R. It looks like following: dr=(theta1-theta2*r) + theta3*sqrt(r)*dW. THe method is inplemented in sde packege that accompanies the book of Iacus "Option Pricing and Estimation of Financial Models…
1
vote
1 answer

Taking out random times of a wiener process

First of all I'm new to matlab and this forum so excuse my ignorance. I generate a standard wiener process in the following way (I made it up myself, so if it's stupid or wrong I would like to know). s =0.0001; % stepsize t = [0:s:T]; % divide…
htd
  • 311
  • 1
  • 5
  • 15
0
votes
2 answers

Poisson random variables with QuantLib

Well hello there, could anyone please tell me if there is a random number generator for Poisson distributed random variables implemented in QuantLib?If yes, where do I find the code for this?I'm trying to simulate a Jump-Diffusion process and need…
0
votes
2 answers

boost::shared_ptr / QuantLib / stochastic process / path generation

Happy Holidays, everyone ! I'm trying to generate paths of a square root process using the QuantLib/Boost C++ libraries and have encountered what I believe to be an annoying little problem with a fast and simple solution! I'm pretty new to…
0
votes
1 answer

Plotting Several OU processes using ggplot and zoo

I need assistance with adapting my ggplot and zoo code to produce plots similar to those generated by my base R code. I am working on a presentation about stochastic methods and need to plot several Ornstein-Uhlenbeck (OU) processes. These processes…
0
votes
0 answers

Nan's in covariance matrix

I've got a problem connected with covariance matrix used to implement Fractional Brownian Motion using Cholesky's decomposition. Code below creates covariance matrix. for i in range(n): for j in range(n): cov_matrix[i, j] = 0.5 *…
0
votes
0 answers

Q-Learning, chosen action takes place with a probability

In my problem I have predefined state and action spaces but when the agent decides to take an action, this action can take place as desired action can take place partially action not applicable at all So the outcome of the action at each step…