1

I've got an array of u8 which I will be partitioning in a complicated way.

Is it possible to update it from multiple rayon tasks? Sort of like this:

    let mut pixels = vec![0; w * h];

 
    rayon::scope(|s| {

        s.spawn(move |_s| {
            my_fn(&mut pixels);
        }
    }

BTW, I know about into_par_iter, but I can't partition the pixels ahead of time.

I.e. the partitioning scheme will be recursive and will depend on the outcome of some of the tasks.

FYI, I will be passing metadata about the pixels back and forth, which will influence which part of pixels my_fn will be working on, which is not shown here.

I'm getting this error, which prevents me from borrowing pixels afterwards.

    |                                 ------ variable moved due to use in closure
...
999 |             write_image(&pixels)
    |                                   ^^^^^^^ value borrowed here after move

Should I just give up on this design, or is there some way to make it work?

pnadeau
  • 427
  • 5
  • 8
  • "the partitioning scheme will be recursive and will depend on the outcome of some of the tasks." Please explain more about this, especially how you know that no collisions can occur. – PitaJ Dec 15 '22 at 21:33
  • You can use atomics. – Chayim Friedman Dec 15 '22 at 21:37
  • "the partitioning scheme will be recursive and will depend on the outcome of some of the tasks." Some of the regions will exit early because I can take advantage of a mathematical property, but I can't know which regions will have this property a-priori. This is why I'm not just striping or tiling the pixels a head of time. – pnadeau Dec 15 '22 at 22:02

1 Answers1

1

Use an array of atomics. If you never modify the same element from different threads, you can safely use Ordering::Relaxed (I think this is free? I am not an expert on that topic).

If you are unable to change the Vec creation, you can convert &mut [int] to &mut [Atomic] as they are guaranteed to have the same layout. On nightly, you can use from_mut_slice(). On stable, you can create it with little unsafe code:

fn from_mut_slice(v: &mut [i32]) -> &mut [AtomicI32] {
    unsafe { &mut *(v as *mut [i32] as *mut [AtomicI32]) }
}

If you want to avoid even that cost (and assuming there is a cost), you can use UnsafeCell if you never access the same pixel from different threads (but the burden to prove that is on you, as a user of unsafe code). Note that UnsafeCell is not Sync so you need to wrap it with a custom type implementing Sync; on nightly, you can use SyncUnsafeCell.

Chayim Friedman
  • 47,971
  • 5
  • 48
  • 77
  • Thanks. I tried `UsafeCell` and so far I can't get it to work. I always end up either being told that it's been moved or that two closures require access to pixels at the same time. (Rayon relies on two nested closures, which seems to complicate things.) I'm not sure how to make use of it in my scenario. When would be the appropriate time to call `get()` on it? – pnadeau Dec 16 '22 at 14:22
  • @pnadeau On each thread, for each pixel. Don't have a big `UnsafeCell`; have one for each pixel. – Chayim Friedman Dec 18 '22 at 00:32
  • I ended up using unsafe code and sharing the pixel array like this: `pixels: &SyncUnsafeCell<&mut [Option]>` and when I need access to the pixels is use this idiom `pixels.get()` – pnadeau Apr 28 '23 at 11:26