C# Ordered Threads plus Concurrency

Question

I'm building a pipeline cycle with threads. Cycles are Fetch, Decode, Execute, and Write Back. I coded a GUI button which triggers each cycle. The problem I'm having is that the output is all jumbled up when I want it to be in a specific order (F->D->X->W). In addition, I want these threads to run concurrently meaning I can't use locks with monitors. From what I know about monitors and locks, they make it so one thread runs at a time and that won't work for me. Here is a simple outline of my code:

public void nextInstruction() 
{
    // These all trigger at once
    // and run in parallel
    fetchEvent.Set();
    decodeEvent.Set();
    executeEvent.Set();
    wbEvent.Set();
}

public void fetch() 
{
    fetchEvent.waitone();
    // do stuff...
}

public void decode()
{
    decodEvent.waitone();
    // do stuff...
}

public void execute()
{
    exeEvent.waitone();
    // do stuff...
}

public void writeBack()
{
    wbEvent.waitone();
    // do stuff...
}

and current output:

F           D           X           W
F      lda $50

F      sta $2        
            D      lda #$10    
F      lda #$0       
            D      sta $1        
                        X      sta $0        
            D      lda $50
                                    W      sta $0        
                        X      lda #$10    
F      sta $0        
F      lda #$10    
            D      lda $50
                        X      sta $1        
                                    W      sta $0        
F      sta $1        
                                    W      sta $0        
                        X      lda $50
            D      sta $2        
            D      lda #$0       
                        X      lda $50
                                    W      sta $0 

my desired output:

F           D           X           W
F      lda $10

F      sta $0        
            D      lda #$10    
F      lda #$11     
            D      sta $0        
                        X      lda #$10  
F      sta $0          
            D      lda $11     
                        X      sta $0    
                                    W      lda #$10      
F      lda #$10    
            D      sta $0
                        X      lda $11         
                                    W      sta $0        
F      sta $1        
            D      lda #$10 
                        X      sta $0 
                                    W      lda $11               
            D      sta $1
                        X      lda #$10
                                    W      sta $0 

I have my print statements formatted this way. Makes it easier for me to see the ordering. Again each of these print statements--F, D, X, W--are each triggered by a distinct thread. Any input on how to achieve this?

I'm also open to using locks if there is a way to use them I'm unaware of.

Answer 1

Reason is that you set all events at once.

Once main thread wakes them up, the order of execution in which they are scheduled by Windows is undefined, and last thread may start sooner than the first.

Look at the producer-consumer collections, such as BlockingCollection: http://msdn.microsoft.com/en-us/library/dd267312(v=vs.110).aspx

These will allow you to have all four running at the same time, consuming product of previous one and sending output to the next one.

You describe the problem as series of sequential steps, triggered by each input element. In between the processing blocks, you might have objects of some other types and a different number of them, but this is still a producer-consumer pattern.

Producer-consumer in this case is applied to all subsequent pairs of your processors. They are actively running, waiting for input from predecessor and sending output to successor. Once you start thinking about how to manage resources to store those temporary outputs, you naturally come to the idea of queuing.

Draw the diagram of the process, create a single BlockingCollection for each data flow link, and give it some queue in constructor to ensure ordering (if needed). Also, specify limits on the number of items in each, that will be your "buffer size". Consumers will use "GetConsumingEnumerable" method and apply foreach on it - it will automatically block when there's no data in the queue from producer.

Once done, re-check each producer-consumer pair, to make sure that they run at the same speed, on average. If any consumer runs significantly faster, consider merging its code into the predecessor producer, because queuing is useless on that link.

Answer 2

try it this way,

public void Button_OnClick() {
    nextInstruction();
}

public void nextInstruction() 
{
    fetchEvent.Set();
}

public void fetch() 
{
    while(true) {
        fetchEvent.waitone();
        // do stuff...
        decodeEvent.Set();
    }
}

public void decode()
{
    while(true) {
        decodEvent.waitone();
        // do stuff...
        executeEvent.Set();
    }
}

Answer 3

ATTENTION! The following will only work in .NET 4 or newer. Neither Parallel nor ConcurrentDictionary are available in older .NET versions.

With Parallel.Invoke(), you can execute your actions concurrently while taking adavantage of this method not returning until all actions have been completed.

Use a data type like a collection or dictionary where the actions can store their (output) results for later sequential processing.

In the example code below, i used a ConcurrentDictionary. The keys of the ConcurrentDictionary are the actions itself. If you want to process this dictionary somewhere else, it might not be a wise idea to use the actions themselves as keys. In such a case, you should rather implement a public enum (representing the actions) to be used as keys for the dictionary.

Since the actions run concurrently and might access the dictionary at exactly the same time, the thread-safe ConcurrentDictionary type hase been chosen. (An ordinary Dictionary is not thread-safe, and might thus cause sporadic, seemingly random errors.)

public class InstructionCycles
{
    private readonly ConcurrentDictionary<Action, string> _dictActionResults = new ConcurrentDictionary<Action, string>();

    private void fetch()
    {
        // do something and store the result in the dictionary
        _dictActionResults[fetch] = "FetchResult";
    }

    private void decode()
    {
        // do something and store the result in the dictionary
        _dictActionResults[decode] = "DecodeResult";
    }

    private void execute()
    {
        // do something and store the result in the dictionary
        _dictActionResults[execute] = "ExecuteResult";
    }

    private void writeBack()
    {
        // do something and store the result in the dictionary
        _dictActionResults[writeBack] = "WriteBackResult";
    }


    public static void nextInstruction()
    {
        InstructionCycles instrCycles = new InstructionCycles();

        Action[] actions = 
        {
            instrCycles.fetch,
            instrCycles.decode,
            instrCycles.execute,
            instrCycles.writeBack
        };

        Parallel.Invoke(actions);

        // output the results in sequential order

        foreach (Action a in actions)
        {
            Console.Out.WriteLine(instrCycles._dictActionResults[a]);
        }
    }
}

Execute an instruction by calling InstructionCycles.nextInstruction().

Using a static method nextInstruction() (which internally creates an instance of InstructionCycles) allows to run multiple instructions in parallel if desired, since each instruction works with its own results dictionary without interfering with other instructions.

If a static method is not desired and it is also not a requirement to have instructions being executed in parallel, the nextInstruction() can be altered into something like this:

    private readonly object _lockObj = new object();

    public void nextInstruction()
    {
        Action[] actions = 
        {
            fetch,
            decode,
            execute,
            writeBack
        };

        lock (_lockObj)
        {
            _dictActionResults.Clear();

            Parallel.Invoke(actions);

            // output the results in sequential order

            foreach (Action a in actions)
            {
                Console.Out.WriteLine(_dictActionResults[a]);
            }
        }
    }

Note the lock statement. If in any case instructionNext() is called while another thread is already executing instructionNext(), the lock will block the execution of the second thread until the first thread finishes with instructionNext(). As lock object a private object should be chosen which is not accessible from outside the class, which will avoid a number of potential dead-lock scenarios.

Answer 4

This is a demonstration of how to achieve something similar like the Parallel.Invoke() example i have given in my first answer, but this time only using AutoResetEvent.

(ManualResetEvent objects could be used instead of AutoResetEvent, but then the code would need to take care about resetting those in case the nextInstruction() method should be called again.)

Each of the instruction cycle tasks sleeps an arbitrary amount of time ("simulating" different execution times), so to demonstrate the effect of the AutoResetEvents ensuring proper order of outputting the results.

Before waiting for the permission to output the results, all cycles will run concurrently. Only with invoking the WaitOne() method on the respective AutoResetEvent (waiting for permission to output the results), the remainder of the tasks (which is outputting of the results) will be executed in a sequential manner.

public class InstructionCycles
{
    private readonly AutoResetEvent DecodeAllowedToOutputEvent = new AutoResetEvent(false);
    private readonly AutoResetEvent ExecuteAllowedToOutputEvent = new AutoResetEvent(false);
    private readonly AutoResetEvent WriteBackAllowedToOutputEvent = new AutoResetEvent(false);

    // 
    // The InstructionFinishedEvent would not be necessary,
    // if nextInstruction() does not need to wait for the instruction to finish.
    //
    AutoResetEvent InstructionFinishedEvent = new AutoResetEvent(false);

    private void fetch()
    {
        try
        {
            // do something useful...
            // For demo purpose, lets just sleep some arbitrary time
            Thread.Sleep(500);

            // This is the 1st cycle.
            // So we don't need to wait for a previous cycle outputting its result.
            Console.Out.WriteLine("FetchResult");
        }
        finally
        {
            // Allow the next cycle to output its results...
            DecodeAllowedToOutputEvent.Set();
        }
    }

    private void decode()
    {
        try
        {
            // do something useful...
            // For demo purpose, lets just sleep some arbitrary time
            Thread.Sleep(200);

            // Processing done.
            // Now wait to be allowed to output the result.
            DecodeAllowedToOutputEvent.WaitOne();

            Console.Out.WriteLine("DecodeResult");
        }
        finally
        {
            // Allow the next cycle to output its results...
            ExecuteAllowedToOutputEvent.Set();
        }
    }

    private void execute()
    {
        try
        {
            // do something useful...
            // For demo purpose, lets just sleep some arbitrary time
            Thread.Sleep(300);

            // Processing done.
            // Now wait to be allowed to output the result.
            ExecuteAllowedToOutputEvent.WaitOne();

            Console.Out.WriteLine("ExecuteResult");
        }
        finally
        {
            // Allow the next cycle to output its results...
            WriteBackAllowedToOutputEvent.Set();
        }
    }

    private void writeBack()
    {
        try
        {
            // do something useful...
            // For demo purpose, lets just sleep some arbitrary time
            Thread.Sleep(100);

            // Processing done.
            // Now wait to be allowed to output the result.
            WriteBackAllowedToOutputEvent.WaitOne();

            Console.Out.WriteLine("WriteBackResult");
        }
        finally
        {
            // Signal that the instruction (including outputting the result) has finished....
            InstructionFinishedEvent.Set();
        }
    }


    public void nextInstruction()
    {
        //
        // The order in which the cycles are started doesn't really matter,
        // since the way how the AutoResetEvents are being used will ensure
        // correct sequence of outputting results.
        //

        Task.Factory.StartNew(fetch);
        Task.Factory.StartNew(decode);
        Task.Factory.StartNew(execute);
        Task.Factory.StartNew(writeBack);

        // 
        // The InstructionFinishedEvent would not be necessary,
        // if nextInstruction() does not need to wait for the instruction to finish.
        //
        InstructionFinishedEvent.WaitOne();
    }
}

By utilizing try {...} finally {...} it is ensured that the chain of events works unhindered even if some code in one of the threads decides to throw an exception.

Even if you dare to deliberately dispose one or more of the AutoResetEvents and then call nextInstruction(), execution of the instruction cycles will still happen concurrently. However, output of the results will not happen anymore as expected, since when one of the cycles tries to wait on the disposed and now invalid AutoResetEvent, an exception will be thrown, and the next thread will be signaled permission to output its results (thanks to the workings of the try-finally block).

Note: On first glance, the code might look similar to inquisitive's answer. However, there are differences in the program flow / event handling which are significant regarding the behavior of the overall process.

Also note, that nextInstruction() as given above is not thread-safe itself. To allow multiple calls of nextInstruction() from different calling threads, like in my other answer a lock has to be utilized to ensure that only one instruction is being executed at a time:

    private readonly object _lockObj = new object();

    public void nextInstruction()
    {
        //
        // The order in which the cycles are started doesn't really matter,
        // since the way how the AutoResetEvents are being used will ensure
        // correct sequence of outputting results.
        //

        lock (_lockObj)
        {
            Task.Factory.StartNew(fetch);
            Task.Factory.StartNew(decode);
            Task.Factory.StartNew(execute);
            Task.Factory.StartNew(writeBack);

            // 
            // When using lock, InstructionFinishedEvent must
            // be used to ensure that nextInstance() remains
            // in the lock until the instruction finishes.
            //
            InstructionFinishedEvent.WaitOne();
        }
    }