Nested loops with a Rust `Sender` and the Rayon's `for_each_with()`

Question

I have some multithreaded code that involves using a nested loop, where the inner one is run in parallel. "Shared" across each thread is a Sender that will return results. Notably Sender implements Send so there should be no problem cloning it and sending it using Rayon's for_each_with(). However, compiling this code:

use std::sync::mpsc::channel;
use rayon::prelude::*;

fn main(){
    let (sender, receiver) = channel();

    (0..5).for_each(|i|{
        (0..5).into_par_iter().for_each_with(&sender, |sender, j|{
            sender.send(i + j).unwrap();   
        });
    });
}

Gives me:

8 |         (0..5).into_par_iter().for_each_with(&sender, |sender, j|{
  |                                              ^^^^^^^ `Sender<_>` cannot be shared between threads safely
  |
  = help: the trait `Sync` is not implemented for `Sender<_>`
  = note: required because of the requirements on the impl of `Send` for `&Sender<_>`

(Playground)

Now I was thinking this might be because I'm trying to clone a reference, but if I move the actual sender into for_each_with() (ie for_each_with(sender, ...)), it will be consumed by the first iteration of the outer loop:

error[E0507]: cannot move out of `sender`, a captured variable in an `FnMut` closure

(Playground).

How can I implement this pattern in a way that satisfies the Rust compiler?

Answer 1

AFAIK, Rayon use a threadpool, this mean that rayon require the item to implement Send cause it will send it to a thread. rayon doesn't clone for each item but for each thread:

fn for_each_with<OP, T>(self, init: T, op: OP) where
    OP: Fn(&mut T, Self::Item) + Sync + Send,
    T: Send + Clone,

We can see the OP take a &mut T not a T. This mean for_each_with() clone for each number of thread used not for the number of item produce.

Reference need to implement Sync to implement Send. Sender is define to not implement Sync. I don't know the detail of this choice anyway this mean &Sender can't be share between thread. I don't think there is solution to remove this constrain.

But if you want you could use crossbeam-channel that implement Sync:

use crossbeam_channel::unbounded; // 0.5.1
use rayon::prelude::*; // 1.5.1

fn main() {
    let (sender, _receiver) = unbounded();

    for i in 0..5 {
        (0..5).into_par_iter().for_each_with(&sender, |sender, j| {
            sender.send(i + j).unwrap();
        });
    }
}

would compile fine. Bonus is that crossbeam-channel claim to be faster. That said, cloning is totally fine for the std Sender:

use rayon::prelude::*;
use std::sync::mpsc::channel;

fn main() {
    let (sender, _receiver) = channel();

    for i in 0..5 {
        let sender = sender.clone();
        (0..5).into_par_iter().for_each_with(sender, |sender, j| {
            sender.send(i + j).unwrap();
        });
    }
}

That would indeed clone O(n) time more but Sender from std is mean to be clone a lot. (Actually it's probably just add one clone cause you choice to make a nested loop, the code probably doesn't clone the last one but just give it the last thread and so you just clone one too much as we can test)

Anyway, all your problems come from a strange situation, one should flatten the iteration like:

use rayon::prelude::*;
use std::sync::mpsc::channel;

fn main() {
    let (sender, _recever) = channel();

    (0..5)
        .into_par_iter()
        .flat_map_iter(|i| (0..5).map(move |j| (i, j)))
        .for_each_with(sender, |sender, (i, j)| {
            sender.send(i + j).unwrap();
        });
}

You could also consider not using Sender rayon is probably mean to be use with collect() and instead of sending the item you could just collect them at the end.

Answer 2

Sender is made to be cloned, its literally in the docs. Each child thread has to have its own sender.