1

For some reason, it appears code inside of the Consumer nor Producer Tasks is ever executed. Where am I going wrong?

using System.Threading.Channels;

namespace TEST.CHANNELS
{
    public class Program
    {
        public static async Task Main(string[] args)
        {
            var channel = Channel.CreateUnbounded<int>();
            var cancel = new CancellationToken();
            await Consumer(channel, cancel);
            await Producer(channel, cancel);

            Console.ReadKey();
        }

        private static async Task Producer(Channel<int, int> ch, CancellationToken cancellationToken)
        {
            for (int i = 0; i < 59; i++)
            {
                await Task.Delay(1000, cancellationToken);
                await ch.Writer.WriteAsync(i, cancellationToken);
            }
        }
        
        private static async Task Consumer(Channel<int, int> ch, CancellationToken cancellationToken)
        {
            await foreach (var item in ch.Reader.ReadAllAsync(cancellationToken))
            {
                Console.WriteLine(item);
            }
        }
    }
}
Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
LCaraway
  • 1,257
  • 3
  • 20
  • 48

2 Answers2

3

If you are new, I recommend reading Tutorial: Learn to debug C# code using Visual Studio. You should know how to put breakpoints to see your code running step-by-step.

Now however since this one involves async/Task, it may looks confusing, but when you step in Consumer, you will see it stops at await foreach (var item in ch.Reader.ReadAllAsync(cancellationToken)) line.

The reason is the consumer is waiting for something that producer never puts in. The reason is your first await put a stop to your code so the 2nd line never get executed.

await Consumer(channel, cancel);
await Producer(channel, cancel);

This should fix the issue:

var consumerTask = Consumer(channel, cancel);
var producerTask = Producer(channel, cancel);

await Task.WhenAll(consumerTask, producerTask);

What the above code says is,

  1. Run Consumer Task, don't wait for it, but keep track of it in consumerTask.

  2. Run Producer Task, don't wait for it, but keep track of it in producerTask.

  3. Wait until both consumerTask and producerTask finishes using Task.WhenAll.

Note that it seems you still have a logical problem with Consumer, since it will never exit so your ReadKey() will likely not getting hit (your app would stuck at the WhenAll line). I think it's much easier "practice" for you if you intend to fix it if it's a bug.

Luke Vo
  • 17,859
  • 21
  • 105
  • 181
  • 1
    [Example on Fiddle](https://dotnetfiddle.net/znD9cM). – Theodor Zoulias Jan 11 '22 at 16:55
  • @TheodorZoulias cool! thanks a lot, I keep forgetting we C# developers also have Fiddle. Unfortunately ImplicitUsings doesn't seem to be supported yet so my code wouldn't run due to missing imports. Hope they add that soon. They do support Top-level Program statement as well – Luke Vo Jan 11 '22 at 16:57
  • 1
    Luke yeap, dotnetfiddle.net is not perfect, but in most cases it gets the job done. :-) – Theodor Zoulias Jan 11 '22 at 16:59
  • Thanks, This was a very helpful way to explain it, both the answer and the fiddle example.. Async/Await is a new concept to me and, while I was aware of 'Task.WhenAll()' I was not sure when to use it. It appears that I have found the case for it. Lesson well learned. I am still trying to wrap my head around debugging across multiple threads... – LCaraway Jan 11 '22 at 17:02
  • 1
    @LCaraway good luck! Async/Await is difficult, I still have problems with it after so many years. One example `WhenAll` can be useful is when you want to do multiple independent `Task`s in the background, for e.g. to request all 3 independent APIs before you can proceed with something else. You may however have trouble if you try it with EF Core `DbContext` so you need to know what is thread-safe and what is not. [This](https://learn.microsoft.com/en-us/visualstudio/debugger/debug-multithreaded-applications-in-visual-studio?view=vs-2022) may also be helpful if you haven't known. – Luke Vo Jan 11 '22 at 17:06
  • Additionally, the ReadKey() was included as a naïve attempt at debugging (in addition to breakpoints) . I see in the Fiddle Example from @TheodorZoulias comment that they added a ch.Writer.Complete(). I assume this closes the channel, would that then flag the Consumer Task to complete? Or do i need code to handle that? – LCaraway Jan 11 '22 at 17:08
  • 2
    @LCaraway I assume so. I have not used `Channel` API before but it looks like it. Without it, your `Consumer` is stuck at `ReadAllAsync` and wouldn't exit the function and the Task wouldn't finish. I got to learned something for myself as well. Win-win! – Luke Vo Jan 11 '22 at 17:09
  • 2
    @LCaraway using channels isn't that difficult if you follow some idioms specific to them. That's why Go uses them so much. A Channel is more than just an asynchronous BlockingCollection – Panagiotis Kanavos Jan 11 '22 at 17:15
2

Your code is trying to consume all messages in the channel before any are produced. While you can store the producer/consumer tasks instead of awaiting them, it's better to use idioms and patterns specific to channels.

Instead of using a Channel as some kind of container, only expose and share Readers to a channel created and owned by a consumer. That's how Channels are used in Go.

That's why you can only work with a ChannelReader and a ChannelWriter too:

  • a ChannelReader is a ch -> in Go, the only way to read from a channel
  • a ChannelWriter is a ch <- in Go, the only way to write.

Using Owned channels

If you need to process data asynchronously, do this in a task inside the producer/consumer methods. This makes it a lot easier to control the channels and know when processing is finished or cancelled. It also allows you to construct pipelines from channels quite easily.

In your case, the producer could be :

public ChannelReader<int> Producer(CancellationToken cancellationToken)
{
    var channel=Channel.CreateUnbounded<int>();
    var writer=channel.Writer;
    _ = Task.Run(()=>{
        for (int i = 0; i < 59; i++)
        {
            await Task.Delay(1000, cancellationToken);
            await writer.WriteAsync(i, cancellationToken);
        }
    },cancellationToken)
   .ContinueWith(t=>writer.TryComplete(t.Exception));

   return channel;
}

The consumer, if one is lazy, can be :

static async Task ConsumeNumbers(this ChannelReader<int> reader, CancellationToken cancellationToken)
    {
        await foreach (var item in reader.ReadAllAsync(cancellationToken))
        {
            Console.WriteLine(item);
        }
    }

Making this an extension method Both can be combined with :


await Producer(cancel)
     .ConsumeNumbers(cancel);

In the more generic case, a pipeline block reads from a channel and returns a channel :

public ChannelReader<int> RaiseTo(this ChannelReader<int> reader, double pow,CancellationToken cancellationToken)
{
    var channel=Channel.CreateUnbounded<int>();
    var writer=channel.Writer;
    _ = Task.Run(async ()=>{
        await foreach (var item in reader.ReadAllAsync(cancellationToken))
        {
            var newItem=Math.Pow(item,pow);
            await writer.WriteAsync(newItem);
        }
    },cancellationToken)
   .ContinueWith(t=>writer.TryComplete(t.Exception));

   return channel;
}

This would allow creating a pipeline of steps, eg :

await Producer(cancel)
      .RaiseTo(0.3,cancel)
      .RaiseTo(3,cancel)
      .ConsumeNumbers(cancel);

Parallel processing

It's also possible to use multiple tasks per block, to speed up processing. In .NET 6 this can be done easily with Parallel.ForEachAsync :

public ChannelReader<int> RaiseTo(this ChannelReader<int> reader, double pow,CancellationToken cancellationToken)
{
    var channel=Channel.CreateUnbounded<int>();
    var writer=channel.Writer;

    _ = Parallel.ForEachAsync(
            reader.ReadAllAsync(cancellationToken),
            cancellationToken,
            async item=>
            {
                var newItem=Math.Pow(item,pow);
                await writer.WriteAsync(newItem);
            })
   .ContinueWith(t=>writer.TryComplete(t.Exception));

   return channel;
}

Beware the order

A Channel preserves the order of items and read requests. This means that a single-task step will always consume and produce messages in order. There's no such guarantee with Parallel.ForEachAsync though. If order is important you'd have to add code to ensure messages are emitted in order, or try to reorder them with another step.

Panagiotis Kanavos
  • 120,703
  • 13
  • 188
  • 236
  • I wish I could accept multiple answers as correct. I am still reading through your answer but looks very compelling. I appreciate the reference to GO as 1) it is another language I like to program in and 2) it is known for its excellence in concurrency and channels. – LCaraway Jan 11 '22 at 17:20
  • 1
    @LCaraway the best guide for C# channels are the `Concurrency in Go` presentations and book. If you read them you'll recognize I use the same techniques, for the same reasons. With owned channels you don't have to worry who gets to cancel what, or when a channel completes. Changing from unbounded to bounded channels is easy too, without modifying anything outside a step method. It's also easier to add error handling if you do it inside each step. You can catch errors in the loop and either skip the bad message or terminate the pipeline – Panagiotis Kanavos Jan 11 '22 at 17:35
  • 1
    @LCaraway In Go tuples are used to return either a value or an error, to allow subsequent steps to handle bad messages. The same can be done in C#. And just like Go, if you need to abort a step you'd need a way to signal upstream methods to cancel. In C# this can be done if each step method was able to signal the CancellationTokenSource used by other steps. Why reinvent the wheel when you can "borrow" the good ideas in Go? – Panagiotis Kanavos Jan 11 '22 at 17:39
  • Under the extension paradigm above, is ConsumeNumbers extending ChannelReader? – LCaraway Jan 13 '22 at 15:25
  • Specifically, I am seeing the following error when trying to implement in a separate class. `Extension method can only be declared in non-generic, non-nested static class` – LCaraway Jan 13 '22 at 15:32
  • Extension methods don't really extend a type, they're syntactic sugar for a static method call, eg `MyStaticClass.ConsumeNumbers(reader,...)`. All LINQ methods are extension methods on `IEnumerable<>`. They don't modify the interface itself. And yes, they must be defined on a static class – Panagiotis Kanavos Jan 13 '22 at 15:33
  • Thanks, I see the dumb thing that I did. I Made the method static, but not the class. – LCaraway Jan 13 '22 at 15:36
  • Under the pipeline model, does this imply that the producer and consumer are running in parallel? Want to make sure I understand the intended processing flow here. – LCaraway Jan 14 '22 at 19:39