Producer consumer collection with ability to read and write batches of data

Question

I'm looking for a collection like BufferBlock

but with methods like:

SendAsync<T>(T[])
T[] ReceiveAsync<T>()

Is anyone can help with?

Answer 1

These methods aren't available, SendAsync<T> only takes a single T and RecieveAsync<T> only returns a single T, not arrays.

SendAsync<T>(T[])
T[] ReceiveAsync<T>()

However there is TryReceiveAll<T>(out IList<T> items) and you can call SendAsync<T> in a loop to send an array into the BufferBlock or write your own extension method, something like this:

public static async Task SendAllAsync<T>(this ITargetBlock<T> block, IEnumerable<T> items)
{
    foreach(var item in items)
    {
         await block.SendAsync(item)
    }
}

Note that SendAsync does return a bool indicating acceptance of the messages, you could return an array of booleans or just return if any of them come back false but that's up to you.

Likely it would be easier to use a BatchBlock<T> that you can send items to as singles using a loop but emits the items in batches, which would be easier than using TryRecieveAll if you're building a pipeline. BatchBlock Walkthrough and BatchBlock Example

Answer 2

ReceiveAsync and SendAsync are available as extension methods on the ISourceBlock and ITargetBlockT<> interfaces. This means that you have to cast the block to those interfaces in order to use the extension methods, eg :

var buffer=new BufferBlock<string>();

var source=(ISourceBlock<string>)buffer;
var target=(ITargetBlock<string>)buffer;

await target.SendAsync("something");

Typically that's not a problem because all Dataflow methods accept interfaces, not concrete types, eg :

async Task MyProducer(ITargetBlock<string> target)
{
    ...
    await target.SendAsync(..);
    ...
    target.Complete();
}

async Task MyConsumer(ISourceBlock<string> target)
{
    ...
    var message=await target.ReceiveAsync();
    ...
}

public static async Task Main()
{
    var buffer=new BufferBlock<string>();
    MyProducer(buffer);
    await MyConsumer(buffer);
}