How to make parallel processing in R with empty result?

Question

I'd like to make parallel processing in R by using packages 'doParallel' and 'foreach'. And, the idea is to make parallel only computations without any outcomes. What I've found looks like 'foreach' operator always return some kind of result that takes memory in the RAM. So, I need any help to have an empty result for parallel processing loops.

# 1. Packages
library(doParallel)
library(foreach)

# 2. Create and run app cluster
cluster_app <- makeCluster(detectCores())
registerDoParallel(cluster_app)

# 3. Loop with result
list_i <- foreach(i = 1:100) %dopar% {
  print(i)
}

# 4. List is not empty
list_i

# 5. How make loop with empty 'list_i' ?
# TODO: make 'list' equal NULL or NA

# 6. Stop app cluster
stopCluster(cluster_app)

With `.combine = 'c'` if you really want to return "nothing". — F. Privé, Oct 30 '18 at 11:36
Both assumptions are not working on. The first set "NULL" value for all 100 elements. The second from get as a result list of 100 numbers. The question here is to get one value as a result - NA or NULL. — Andrii, Oct 30 '18 at 15:01
`print(x)` actually returns `x` invisibly, e.g. `y <- print(42)` assigns 42 to `y`. So, as suggested above, `y <- foreach(x = 1:3, .combine = c) %dopar% { ... ; NULL }` will return `NULL`. FYI, as rule of thumb, think of `foreach()` as you think of `lapply()` - that is - you want to use them to return values. It's rare you only want them for their side effects alone. Also, make no assumptions in what order the elements are processed. — HenrikB, Oct 30 '18 at 19:42

score 0 · Answer 1 · answered Oct 31 '18 at 07:05

Here is the solution I found:

# 1. Packages
library(doParallel)
library(foreach)

# 2. Create and run app cluster
cluster_app <- makeCluster(detectCores())
registerDoParallel(cluster_app)

# 3. Loop with result
list_i <- foreach(i = 1:100) %dopar% {
  print(i)
}

list_i

# 4. Mock data processing
mock_data <- function(x) {
  data.frame(matrix(NA, nrow = x, ncol = x))
}

# 4. How make loop with empty 'list_i' ?
foreach(i = 1:10, .combine = 'c') %dopar% {

  # 1. Calculations
  mock_data(x)

  # 2. Result
  NULL
}

# The results has only one value 'NULL' (not a data set)
list_i


# 5. Stop app cluster
stopCluster(cluster_app)

I think you got it but you would better provide the same example loop for the case where the result is null. I let you also remark that `print(i)` does not print anything in a loop with %dopar%. — Jean Paul, Nov 26 '20 at 16:46

How to make parallel processing in R with empty result?

1 Answers1