2

I have a loop in Rust, which basically looks like this:

while let Some(next) = myqueue.pop_front() {
   let result = next.activate();
   if result.0 {
      myqueue.extend(result.1.into_iter());
   }
}

I want to paralelize this loop. Naturally, rayon crate came to my mind with the parallel executed for_each loop however the problem is that would require myqueue object to be owned by both the main thread and the child threads since the collection that is being iterated over is being modified by the threads and I do not want to use an unsafe block. Therefore, I got stuck and wanted to help with rayon create over this (or maybe a completely different approach).

Any and all help is welcome.

Thank you!

Fioralba Amaranta
  • 17
  • 2
  • First of all, there is a non-trivial issue about how to prevent data-races. Assuming you achieve parallelism, there is the risk that two pieces of code will try to read/write `my_queue` at the same time, which is bad and which is why Rust will prevent that. You have to decide how to handle that, that is, how will you prevent this terrible situation? You could, for instance, put `my_queue` in a `Mutex`, and only allow a single piece of code to access your queue at a given time, but in this situation that might create a lot of contention. – jthulhu Apr 16 '22 at 14:11
  • @BlackBeans yes exactly however that requires the main thread to own the iterable so that it can iterate over it and because the threads need to push something to the iterable / collection, they would also need to own the object as well and thus the code won't compile. – Fioralba Amaranta Apr 16 '22 at 14:15
  • Had a similar problem, implemented a custom [taskstream.rs](https://github.com/martinxyz/progenitor/blob/3ea2442d316c7f9c45706f012e6011b2818cb3a6/crates/progenitor/examples/map-elites/taskstream.rs#L6-L21) module. I hope there is a better solution, it feels like side-stepping rayon. – maxy Apr 16 '22 at 18:27

0 Answers0