4

I'm curious what the design argument for doParallel:::doParallelSNOW() giving a warning when globals are explicitly specified in .export could be? For example,

library("doParallel")
registerDoParallel(cl <- parallel::makeCluster(2L))

a <- 1
y <- foreach(i = 1L, .export = "a") %dopar% { 2 * a }
## Warning in e$fun(obj, substitute(ex), parent.frame(), e$data) :
##  already exporting variable(s): a

Is it possible to disable above warnings in doParallel?

I'd argue that such a warning would suggest the user/developer not to specify .export and rely fully on foreach adaptors to take care of globals. However, the success of identification of globals differ among adaptors.

Package versions: doParallel 1.0.11, iterators 1.0.9, foreach 1.4.4

CLARIFICATION 2018-01-14: These warnings occur after doParallel first having identified globals automatically and then appending the ones the developer list in .export - my question is on why it doesn't not accept the deliberate choice that was done by the developer? Warning about correct globals is a bit misleading.

HenrikB
  • 6,132
  • 31
  • 34

2 Answers2

0

I may have a solution but I wouldn't recommend to do what you're asking in the first place.

Anyway, if you enclose your foreach call in another function, a wouldn't be found in the direct environment of the foreach call so that it wouldn't be exported automatically.

library("doParallel")
registerDoParallel(cl <- parallel::makeCluster(2L))

a <- 1
foreach2 <- function() {
  foreach(i = 1L, .export = "a") %dopar% { 2 * a }
}
y <- foreach2()
F. Privé
  • 11,423
  • 2
  • 27
  • 78
  • A real use case is `caret`, which does not specify `.export`. (I'm not sure if this is because caret is fine tuned for doParallel only, or that is the official foreach approach - "don't specify `.export`"). If [one specify globals explicitly](https://github.com/HenrikBengtsson/caret/commit/54fc61a331fde4cb9910c06cfc6393d1d40b4939), then one get the warning in my original question. – HenrikB Jan 14 '18 at 19:13
  • Thanks Florian, I've reread my original question and realized it was not clear - I've updated with a clarification at the end. Yes, tricking `foreach::getexports()` to fail to detect `a` as a global would avoid the warning and I agree it's not a good idea. – HenrikB Jan 14 '18 at 19:17
0

My solution is simply remove .export = "a" then the warning gone since the warning just said .export redundantly exports the a object.

Xiaorui Zhu
  • 161
  • 7
  • The problem with that approach is **foreach** fails to find some globals in some cases, which means it's a trial and error game. Unfortunately, this also depends on whether you put the code in a script or in a package, so there's no one-solution fits everything. The main reason for my question was to inquire on the design choice and whether it was intentional or not. FWIW, there's been discussion to modernize how **foreach** finds globals, cf. https://github.com/RevolutionAnalytics/foreach/issues/2 – HenrikB Jun 16 '21 at 17:30