0

I have to run multiple simulations of the models. I am trying to run different simulations on different cores and save the results to global env. I have the below code which isn't working. For example I have to run xgboost three times ( with all same parameters, just 3 re-runs and I want to run the three simulations on different cores and return the imp_df_i to global env.

cores=detectCores()
cl <- makeCluster(cores[1]-1) #not to overload your computer
registerDoParallel(cl)

finalMatrix <- foreach(i=1:3, .combine=cbind) %dopar% {
require(xgboost)

  xgb_1 = xgboost(dtrain,
                  params = xgb_params_1,
                  nrounds = 3,                                                 # max number of trees to build
                  verbose = TRUE,                                         
                  print_every_n = 1,
                  early_stop_round = 10)                                          # stop if no improvement within 10 trees

  col_names = attr(tr.x, ".Dimnames")[[2]]
  imp = xgb.importance(col_names, xgb_1)               
  imp_df=data.frame(imp)
  assign(paste("imp_df",i, sep = "_"),imp_df,envir = .GlobalEnv )

  }
#stop cluster
stopCluster(cl)
pfx
  • 20,323
  • 43
  • 37
  • 57
Learner_seeker
  • 544
  • 1
  • 4
  • 21
  • Why do you want to do this? xgboost runs in parallel. Just specify the number of threads via nthread parameter and run your 3 re runs via a for loop. – phiver Sep 04 '18 at 10:35
  • @phiver I found significant improvements when running xgboost in parallel compared to running it sequential when tuning hyper parameters. In both cases `nthread` was set to default (available threads). When not running in parallel the core load is about 20-30%, when in parallel they go up to 90-100%. – missuse Sep 04 '18 at 11:00
  • Like missuse mention - i ran a simple for loop to run the three simulations, i want to compare time if ran parallel in 3 cores. My assumption is also that it will speed up. However i am not able to get around the correct syntax. – Learner_seeker Sep 04 '18 at 12:11

0 Answers0