I used to do parallel computing with R, in which I had three computers connected on LAN; two windows 11 computers with wsl2 (ubuntu 22.04), and one ubuntu computer. The main computer runs an R script which controlled other two computers as slaves, each has 16 or 8 logical cores. Since the main computer has 16 logical cores, I used to run 36 different sets of parameters simultaneously: 14 at the main, 15 and 7 at each slave computer.
The script I used to use was
1 │ node1 <- "192.168.1.10" # A-wsl
2 │ node2 <- "192.168.1.11" # B-wsl
3 │ node3 <- "192.168.1.12" # C
4 │ cl <- c(node1, node2, node3)
9 │ custom_workers <- function() {
10 │ switch(Sys.info()[["nodename"]],
11 │ "A" = 14L,
12 │ "B" = 15L,
13 │ "C" = 7L,
14 │ )
22 │ plan(list(
23 │ tweak(cluster, workers = cl),
24 │ tweak(multisession, workers = custom_workers)
25 │ ))
Then I called future_map twice with nested structure, first allocating parameters for each computer, and then called again in each machine to allocate parameters for each node.
This worked just fine until recently. However, I found the script does not work now, and interestingly, the master machine which runs source R script under wsl2 can control a slave wsl2 machine without a problem, but the master machine seems not communicate to the ubuntu slave machine, and future_map seems not send any data or script to the machine.
I ran the script on the main machine, and the wsl2 slave machine showed all the cores were being used, but the ubuntu machine showed no sign of using cores. As a result the script stops with error message, when the output from ubuntu machine are to be received.
Interestingly, I can control the ubuntu machine with ssh from the main machine. So, I have no idea about what is happening here. All I can remember is that this happened after I updated my windows 11 this fall and switched using systemd from genie's version to wsl2's native version, though I am not sure about this.
I need your help. Any suggestion would be greatly appreciated. Thank you.
Hiroto
