1

I am reading in an XML file and want to try and improve the performance using foreach and doSNOW, however when using plyr I encounter an error. I think it might be because I have missed a way to send the functions and packages to the workers.

library(foreach)  
library(doSNOW)  
cl <- makeCluster(4)  
registerDoSNOW(makeCluster(4, type = "SOCK"))

> getDoParWorkers()  
[1] 4  
> getDoParName()  
[1] "doSNOW"  
> getDoParVersion()  
[1] "1.0.5"

namelist = list()  
out1 <- foreach(i = 1:xmlSize(root[[3]])) %dopar% {  
     namelist[[i]] <- llply(xmlToList(root[[3]][[i]][[2]]), 
                        data.frame, .parallel = TRUE)
}

out2 <- t(as.vector(as.data.frame(namelist)))

Generates the error:

Error in { : task 1 failed - "could not find function "llply""
csgillespie
  • 59,189
  • 14
  • 150
  • 185
Sam35
  • 35
  • 7
  • 2
    It would be nice if you could repost it as a self-contained example once it works. What is root, and it certainly requires a library(XML). – Dieter Menne May 04 '12 at 15:49

1 Answers1

4

?foreach then look at the .packages argument

GSee
  • 48,880
  • 13
  • 125
  • 145
  • 1
    +1. Also, @Sam35, your fourth line should be `registerDoSNOW(cl)`. Furthermore, there's no need to assign to `namelist` (it won't work anyway). Instead, let the results of `llply` go directly to `out1`, which they will if you take out `namelist[[i]] <- `. – BenBarnes May 04 '12 at 14:19