I have written a function with 3 nested foreach loops, running in parallel. The goal of the function is to split a list of 30 [10,5] matrices (i.e. [[30]][10,5]
) into a list of 5 [10,30] matrices (i.e. [[5]][10,30]
).
However, I am trying to run this function with 1,000,000 paths (i.e. foreach (m = 1:1000000)
), and obviously, the performance is terrible.
I'd like to avoid apply functions if possible because I've found that they don't work well when used in conjunction with parallel foreach loops:
library(foreach)
library(doParallel)
# input matr: a list of 30 [10,5] matrices
matrix_splitter <- function(matr) {
time_horizon <- 30
paths <- 10
asset <- 5
security_paths <- foreach(i = 1:asset, .combine = rbind, .packages = "doParallel", .export = "daily") %dopar% {
foreach(m = 1:paths, .combine = rbind, .packages = "doParallel", .export = "daily") %dopar% {
foreach(p = daily, .combine = c) %dopar% {
p[m,i]
}
}
}
df_securities <- as.data.frame(security_paths)
split(df_securities, sample(rep(1:paths), asset))
}
Overall, I'm trying to convert this data format:
[[30]]
[,1] [,2] [,3] [,4] [,5]
[1,] 0.2800977 2.06715521 0.9196326 0.3560659 1.36126507
[2,] -0.5119867 0.24329025 0.1513218 -1.2528092 -0.04795098
[3,] -2.0293933 -1.17989270 0.3053376 -0.9528611 0.86758140
[4,] -0.6419024 -0.24846720 -0.6640066 -1.7104961 -0.32759406
[5,] -0.4340359 -0.44034013 3.3440507 0.7380613 2.01237069
[6,] -0.6679914 -0.01332117 1.9286056 -0.7194116 0.15549978
[7,] 0.5919820 0.11616685 -0.8424634 -0.7652715 1.34176688
[8,] 0.8079152 0.40592119 -0.4291811 0.9358829 -0.97479314
[9,] -0.0265207 -0.03598320 1.1287344 0.4732984 1.37792596
[10,] 1.0553966 0.65776721 -1.2833613 -0.2414846 0.81528686
To this format (obviously up to V30):
$`5`
V1 V2 V3 V4 V5 V6 V7
result.2 -0.11822260 1.7712833 1.97737285 -1.6643193 0.4788075 1.2394064 1.4800787
result.7 -1.23251178 0.4267885 -0.07728632 0.3463092 0.8766395 0.6324840 0.5946710
result.2.1 -1.27309457 -0.3128173 -0.79561297 -0.4713307 -0.4344864 0.4688124 -0.5646857
result.7.1 0.51702719 -1.6242650 -2.37976199 -0.1088408 0.4846507 -0.7594376 0.9326529
result.2.2 1.77550390 0.9279155 0.26168402 0.4893835 1.4131326 0.5989508 -0.3434010
result.7.2 -0.01590682 -0.5568578 1.35789122 -0.1385092 -0.4501515 -0.2581724 0.5451699
result.2.3 0.30400225 -1.0245640 -0.05285694 -0.1354228 0.3070331 -0.7618850 1.0330961
result.7.3 -0.08139912 0.4106541 1.40418839 0.2471505 1.2106539 1.3844721 0.4006751
result.2.4 0.94977544 -0.8045054 1.48791211 1.4361686 -0.3789274 -1.9570125 -1.6576634
result.7.4 0.70449194 1.6887800 0.56447340 0.6465640 2.6865388 -0.7367524 0.6242624
V8 V9 V10 V11 V12 V13
result.2 -0.432404728 -1.6225350 0.09855465 0.17371907 0.3081843 0.15148452
result.7 -0.597420706 0.6173004 0.07518596 2.01741406 0.1767152 -0.39219471
result.2.1 0.918408322 -1.6896424 -0.13409626 0.38674224 0.3491750 -1.61083286
result.7.1 2.564057340 -0.7696399 1.06103614 1.38528367 1.1684045 -0.08467871
result.2.2 0.951995816 0.1910284 1.79943500 2.13909498 0.2847664 0.31094568
result.7.2 -0.479349220 -0.2368760 0.04298525 -0.40385960 0.3986555 -1.93499213
result.2.3 -1.382370069 1.0459845 -0.33106323 -0.43362925 0.7045572 -0.30211601
result.7.3 -1.457106442 0.1487447 -2.52392942 -0.02399523 -1.0349746 0.87666365
result.2.4 -0.848879365 0.7521024 0.16790915 0.47112444 0.8886361 -0.12733039
result.7.4 -0.003350467 0.4021858 -1.80031445 -1.42399232 1.0507765 -0.36193846