Parallel/async code with C#'s Span-like type

Question

Span<T> and Span2D<T> (from the High Performance Community Toolkit) are ref struct types. This means they can't be passed to lambdas, local functions, or async contexts.

However, consider the following example using a Span2D:

namespace Example;

public static class ExampleMethod
{
    public static void TransformExample<T>(Span2D<T> data)
    {
        // Breakdown the problem into 4 quadrants and execute them in parallel
        var halfRows = data.Height / 2;
        var halfCols = data.Width / 2;
        
        TransformQuadrant(data[..halfRows, ..halfCols]); // Top-left quadrant
        TransformQuadrant(data[..halfRows, halfCols..]); // Top-right quadrant
        TransformQuadrant(data[halfRows.., ..halfCols]); // Bottom-left quadrant
        TransformQuadrant(data[halfRows.., halfCols..]); // Bottom-right quadrant   
    }
    
    private static void TransformQuadrant<T>(Span2D<T> data)
    {
        // Do something interesting with this quadrant of `data`
    }
}

If you try to wrap those 4 calls in some kind of async context, either by making TransformQuadrant async, or - for example - using:

var tasks = new List<Task>
{
    Task.Run(() => TransformQuadrant(data[..halfRows, ..halfCols])),
    Task.Run(() => TransformQuadrant(data[..halfRows, halfCols..])),
    Task.Run(() => TransformQuadrant(data[halfRows.., ..halfCols])),
    Task.Run(() => TransformQuadrant(data[halfRows.., halfCols..]))
};

Task.WaitAll(tasks);

You're greeted with

Error CS1628: Cannot use ref, out, or in parameter 'data' inside an anonymous method, lambda expression, query expression, or local function

Which says, from the MS docs:

CS1628: Cannot use in, ref, or out parameter inside an anonymous method, lambda expression, or query expression.

So the question is how can one make this type of work parallel/async? I understand Memory<T> but that implies paying for the conversion to Span<T>, so I wonder how this can be done without these overheads.

Related questions: - https://stackoverflow.com/questions/59605908/what-is-a-working-alternative-to-being-unable-to-pass-a-spant-into-lambda-expr - https://stackoverflow.com/questions/66746744/how-can-i-do-operations-on-spant-in-parallel In summary: `ref struct` isn't allowed on the heap. You can make a hacky workaround by using unsafe pointers but in my opinion the best option would to use Memory2D. Unrelated; if you don't want to make multiple Task instances, you can also use `Parallel.Invoke` instead of `Task.WaitAll`. — gerard, May 16 '23 at 09:17
Another option is to supply a callback delegate that allows the tasks to pick up their spans synchronously (on the same "thread") — Charlieface, May 16 '23 at 09:40

score 1 · Answer 1 · answered May 16 '23 at 16:54

I wonder how this can be done without these overheads

You cannot do it with spans, because spans are only alive on the function stack as a parameter. When you use await, the function ends and a continuation is queued up in the task you get as an output, so without the original function you no longer have the original stack and thus the span.

Same with the lambda stuff you're doing there, you're passing them as continuations to the thread creation code, and by the time they're actually called the original span's backing object might very well not exist anymore.

Granted, the caller might still root the object that's backing your span, but it might also not. Which is unlike Memory<> which actually references (and thus roots) the backing object.

As to solutions, if the original object still exists you can just reference it directly (if a reference type) or through a pointer (for structs, dangerous). And if not, then you need something like Memory<> to keep the reference alive. Honestly if the code you're writing is so heavy as to require 4 separate threads to do their work, lugging around a memory handle is not that big a deal.

score 0 · Answer 2 · answered May 16 '23 at 20:27

One common solution is to provide a callback delegate that can return a Span, so each individual thread can fetch its own Span. Note that this does not help if you are using async as that creates a state machine off the (logical) stack.

public static void TransformExample<T, TArg>(SpanAction<T, TArg> dataFunc, TArg arg)
{
    // Breakdown the problem into 4 quadrants and execute them in parallel
    var halfRows = data.Height / 2;
    var halfCols = data.Width / 2;

    var tasks = new List<Task>
    {
        Task.Run(arg => TransformQuadrant(arg => dataFunc(arg)[..halfRows, ..halfCols])),
        Task.Run(arg => TransformQuadrant(arg => dataFunc(arg)[..halfRows, halfCols..])),
        Task.Run(arg => TransformQuadrant(arg => dataFunc(arg)[halfRows.., ..halfCols])),
        Task.Run(arg => TransformQuadrant(arg => dataFunc(arg)[halfRows.., halfCols..])),
    };

    Task.WaitAll(tasks);
}
    
private static void TransformQuadrant<T, TArg>(SpanAction<T, TArg> dataFunc, TArg arg)
{
    var data = dataFunc(arg);
    // Do something interesting with this quadrant of `data`
}

The new string.Create does something similar. Note that you cannot use normal generic Func delegates for Span either, you need to create your own or use SpanAction.

Parallel/async code with C#'s Span-like type

2 Answers2