0

I want to make my apply() function run parallel. I have loaded the parallel package and made a cluster. Then i have just changed the apply() function to the parApply() function. Unfortunately R doesn't seem to see the function parApply(). Help would be very appreciated.

Kind Regards, Jakub

My initial code:

  • TRAITS and SNP are matrices
  • the calling of apply() worked with no errors
library(parallel)
system.file(package='parallel')
detectCores()

copies_of_r <- 7

cl <- makeCluster(copies_of_r)

parApply(cl ,TRAITS,2,function(x) parApply(cl, SNP, 2, function(y) summary(lm(x~y))$coefficients[2,4]))

I got this:

> library(parallel)
> system.file(package='parallel')
[1] "C:/PROGRA~1/R/R-42~1.2/library/parallel"
> detectCores()
[1] 8
> 
> copies_of_r <- 7
> 
> cl <- makeCluster(copies_of_r)
> 
> parApply(cl ,TRAITS,2,function(x) parApply(cl, SNP, 2, function(y) summary(lm(x~y))$coefficients[2,4]))
Error in checkForRemoteErrors(val) : 
  7 nodes produced errors; first error: nie udało się znaleźć funkcji* 'parApply'`
  • "nie udało się znaleźć funkcji" == "failed to find funtion"

I also tried to call parApply() in this way:

parallel:: parApply(cl ,TRAITS,2,function(x) parApply(cl, SNP, 2, function(y) summary(lm(x~y))$coefficients[2,4]))

but I got the same result as before.

I also tried:

getAnywhere(parApply)

And suprisingly, R seems to see the function parApply() but has a problem, when I try to call it.

A single object matching ‘parApply’ was found
It was found in the following places
  package:parallel
  namespace:parallel
with value

(...)
Qba Liu
  • 51
  • 4
  • What do you get if you run `parallel::parApply()` - no arguments? What about `parallel::parApply` to view the source? – Paul Stafford Allen Jan 27 '23 at 09:11
  • 1
    One possible candidate is that you're trying to call parApply() within a parApply() call - see discussion at https://stackoverflow.com/questions/8413188/can-i-nest-parallelparlapply – Paul Stafford Allen Jan 27 '23 at 09:15
  • As Paul says, you're calling `parApply()` in the parallel workers running in the background. Since you haven't attached the **parallel** package in those workers, they have no idea about that function. You only attached **parallel** in the main R session. – HenrikB Jan 27 '23 at 18:05
  • More importantly, you rarely want to do nested parallelization as you attempt. It risks overloading your machine. Also, you cannot pass one cluster object (here `cl`) down to parallel workers; such objects only work in the R session they're created in. – HenrikB Jan 27 '23 at 18:07

0 Answers0