Stack Overflow
Stack Overflow

Questions tagged [snow]

DO NOT USE FOR SNOW ANIMATION.The R package SNOW (acronym for Simple Network Of Workstations) provides a high-level interface for using a cluster of workstations for parallel computations. Use with the [r] tag.

The package snow (acronym for Simple Network Of Workstations) provides a high-level interface for using a cluster of workstations for parallel computations in R.

snow implements an interface to three different low-level mechanisms for creating a virtual connection between processes:

  • Socket
  • PVM (Parallel Virtual Machine)
  • MPI (Message Passing Interface)

The snowfall package provides a more recent alternative to snow. Functions can be used in sequential or parallel mode.

Resources:

127 questions
2
votes
0 answers

R parallel job hangs

I am running the snow-test.R script as written on the site: https://hpcf.umbc.edu/other-packages/how-to-run-r-programs-on-maya/ The script is run on a cluster using SLURM with the command: mpirun -np 1 R CMD BATCH --no-save --no-restore snow-test.R…
armando
  • 1,360
  • 2
  • 13
  • 30
2
votes
0 answers

Can not makeCluster on windows 10

I can not use parallel::makeCluster or snow::makeCluster on a windows 10 computer. I frequently receive this error, each time with a different port number: Error in socketConnection("localhost", port = port, server = TRUE, blocking = TRUE, : …
Ahmed El-Gabbas
  • 398
  • 3
  • 10
2
votes
1 answer

Can the parallel or snow packages in R interface with a spark cluster?

I am dealing with a computationally intensive package in R. This package has no alternative implementations that interfaces with a Spark cluster;however, it does have an optional argument to take in a cluster created with the parallel package. My…
niccalis
  • 134
  • 1
  • 7
2
votes
0 answers

Parallel estimation of multiple nonparametric models using np and snowfall

I am trying to estimate multiple nonparametric models using snowfall. So far I had no problems, but now I run into a problem that I feel unable to resolve. In the MWE below we simply estimate only one model on one node. In my application the…
Martin Schmelzer
  • 23,283
  • 6
  • 73
  • 98
2
votes
0 answers

Package snow: makeCluster hangs

I have trouble with the initialization of a cluster with the snow package on R 3.4.2 on a Windows machine. When I run the following code, R hangs after the last line without ever setting up the cluster: library(snow) nodes <- rep("localhost",2) cl…
qcd
  • 21
  • 2
2
votes
0 answers

foreach Error: aborting ... Error in unserialize(nodecon) : error reading from connection

Sometimes,my R instance is silently dying.I am using foreach for parallel processing.I am using 16core and 64GB ram machine.I am using following code to execute foreach. cl <- makeCluster(6,outfile=mkclog); registerDoParallel(cl, cores =…
Aashu
  • 1,247
  • 1
  • 26
  • 41
2
votes
0 answers

Error in unserialize(socklist[[n]]) : error reading from connection

I'm using R and I'm running a parallel code on 7 cores on a Linux machine. When using a small dataset, my code takes about 2 hours and works fine. When using a 6x larger dataset, the codes takes much longer (probably because it needs to swap), but…
AF7
  • 3,160
  • 28
  • 63
2
votes
0 answers

R: Working with sparse matrices within snow clusters

I am trying to work with sparse matrices generated by igraph under a parallel cluster made by snow, but it seems that operations such as taking a sum of the matrix or doing something like which(element > 0) don't work even though their non-parallel…
user39530
  • 21
  • 2
2
votes
1 answer

Incorrect number of dimensions - parallel R computation

I have an issue when using the tm package and parallel computation in R and I'm not sure if I'm doing something silly or if it is a bug. I created a small reproducible example: # Load the libraries library(tm) library(snow) # Create a Document Term…
Lorenzo Rossi
  • 1,481
  • 1
  • 9
  • 16
2
votes
1 answer

Max number of workers/slaves for parallel job snow

I'm running a foreach loop with the snow back-end on a windows machine. I have 8 cores to work with. The rscript is exectuted via a system call embedded in a python script, so there would be an active python instance too. Is there any benefit to…
imouellette81
  • 234
  • 2
  • 5
2
votes
0 answers

MPI remote nodes do not execute

I am trying to build a cluster to parallelize a R function. I use Open MPI in Ubuntu and R packages (Rmpi and Snow). The test code I am running is: cl <- makeMPIcluster(8) fun <- function(){ …
user2212505
  • 43
  • 4
2
votes
1 answer

How to set up cluster slave nodes (on Windows)

I need to run thousands* of models on 15 machines (each of 4 cores), all Windows. I started to learn parallel, snow and snowfall packages and read a bunch of intro's, but they mainly focus on the setup of the master. There is only a little…
Tomas
  • 57,621
  • 49
  • 238
  • 373
2
votes
1 answer

Generating random numbers with Snow package in R

I'm currently running a program which is multithreaded via the snow package (using clusterApplyLB) and in the code I generate a random number, but it seems the threads are all using the same seed to generate this number so they give the same…
jpd527
  • 1,543
  • 1
  • 14
  • 30
2
votes
1 answer

How does snow distribute list elements to workers?

How many list elements are sent to each worker process when calling parLapply()? For example, let's say we have a list of 6 elements and 2 workers on a snow SOCK cluster. Does parLapply() sends two list elements to each worker in one send call, or…
Robert Kubrick
  • 8,413
  • 13
  • 59
  • 91
2
votes
2 answers

Parallelization of large simulations over a grid in R

I am running a series of large simulations over a grid. I am implementing the simulations by row and I have found that my sampling functions are a bottleneck. I've attempted to use the foreach and doMC libraries to speed up the process, but I've…
user2059737
  • 183
  • 1
  • 1
  • 8
1 2
3
…
8 9