What are alternatives to `if` statements to improve performance of MCMC algorithms?

Question

I am running a MCMC algorithm with Metropolis Hastings step in R, which requires accepting or rejecting a proposal sample according to a logical rule. Currently, I have implemented this as

if(sample meets condition){accept}
else{reject}

I heard that if statements are slow, but MCMC requires usually evaluating many proposal samples, certainly in the ten thousands. What are faster alternatives to improve the speed of this part of any MCMC algorithm?

To give an example of the step in the code:

A = sample(c(0,1),1,prob=c(1-A,A))
if(A==1){
  s_acc[i,] = s_new
  s_old     = s_new
  acc[i] = 1
}
else{
  s_acc[i,] = s_old
  acc[i] = 0
}

where A is the acceptance probability of a sample, s_new is the value of the proposed sample, s_acc is the vector of all accepted samples, s_old is the current previously accepted sample, and acc is a count of which samples were accepted. This part of the code is embedded in a for loop with a high number of iterations.

Just throwing an idea out there... If you have a way to convert your conditions to a numeric value, you could use a dictionary/hashtable of known-good values. Accept or reject based on whether it exists in the known-good collection. You could build the known-good list on startup by pre-evaluating all of the "good" options using the same code as would be used to evaluate your run-time value. — theGleep, Sep 22 '17 at 14:51

score 1 · Answer 1 · answered Sep 22 '17 at 14:50

1

Switch is generally faster than If statements but in both cases these are negligible differences, unless it is really really critical code.

answered Sep 22 '17 at 14:50

Michele Federici

1,755
1
14
17

1

If the evaluation can fit into a switch statement, performance can *probably* be improved by using a dictionary/hashtable with "case" values as keys and handler functions as values. A few years ago, I used this technique in JavaScript to reduce 10seconds code evaluation down to less than 1 second. YMMV – theGleep Sep 22 '17 at 14:53
See my edit. Can `switch` be implemented in this setting (i.e. with multiple arguments following `if`)? – tomka Sep 22 '17 at 14:58
The solution proposed by @theGleep is also frequent and effective in many implementations because of hashmaps' low search time complexity (O(1)) – Michele Federici Sep 24 '17 at 21:12

score 1 · Answer 2 · answered Sep 22 '17 at 15:59

1

I dont think if is slow (although ifelse is, but it's different). What is slow is to iterate over lots of elements in R.

If you really care about performance you could use either vectorization (if the iterations are independent) or Rcpp.

This example might be of interest for you.

answered Sep 22 '17 at 15:59

F. Privé

11,423
2
27
78

Thanks, MCMC iterations usually are dependent. I will look at Rcpp. – tomka Sep 22 '17 at 16:06

score 0 · Answer 3 · answered Jul 22 '21 at 11:06

Are you intending to handwrite this algorithm? Because several R packages (based on Stan, JAGS, BUGS code) can actually do Metropolis-Hastings and the functions are normally optimized for speed, since it is ultimately the limiting factor.

There is a number of R-compatible software out there that will work wonders for you. That is, if automating your processes is an option.

What are alternatives to `if` statements to improve performance of MCMC algorithms?

3 Answers3