Reduce CPU usage while processing large amount of data

Question

I am writing a real time application which receives around 2000 messages per second which was pushed in a queue. I have written a background thread which process the messages in the queue.

private void ProcessSocketMessage()
{
    while (!this.shouldStopProcessing)
    {
        while (this.messageQueue.Count > 0)
        {
            string message;
            bool result = this.messageQueue.TryDequeue(out message);
            if (result)
            {
               // Process the string and do some other stuff
               // Like updating the received message in a datagrid  
            }
        }
    }
}

The problem with the above code is that it uses insane amount of processing power around 12% of CPU(2.40 GHz dual core processor).

I have 4 blocks similar to the one above which literally takes up 50 % of CPU computing power. Is there anything which can be optimized in the above code?

Adding a Thread Sleep of 100 ms before second while loop end does seems to be increase the performance by 50%. But am I doing something wrong?

So why is it using so much CPU? Is it "busy-waiting", i.e. is the Count 0 in most iterations? There really isn't a silver bullet for this, read [ask] and include enough details in your question. — CodeCaster, Sep 14 '17 at 13:18
@CodeCaster: I think so. But I receive more than 200 messages per second. Is there a way to avoid that? — Ramesh Durai, Sep 14 '17 at 13:19
See the answer to https://stackoverflow.com/q/19930320/56778. That question might even be considered a duplicate. Basically, you want to create a simple producer-consumer model that used [BlockingCollection](https://msdn.microsoft.com/en-us/library/dd267312(v=vs.110).aspx). There are many, many examples of such a thing. — Jim Mischel, Sep 14 '17 at 13:22
Probably won't help in performance, but curious do you need a `while` loop on `while (this.messageQueue.Count > 0)`? Can't it be an `if` statement? After all your outer loop is going to continuously loop if `true` — penleychan, Sep 14 '17 at 13:23
*Instead* of busy-waiting or manually handling queues, use the purpose-built ActionBlock class. It has its own input buffer, it doesn't use busy waiting and can even process messages in parallel — Panagiotis Kanavos, Sep 14 '17 at 13:24

score 6 · Answer 1 · answered Sep 14 '17 at 13:43

This functionality is already provided in the Dataflow library's ActionBlock class. An ActionBlock has an input buffer that receives messages and processes them by calling an action for each one. By default, only one message is processed at a time. It doesn't use busy waiting.

void MyActualProcessingMethod(string it)
{
  // Process the string and do some other stuff
}

var myBlock = new ActionBlock<string>( someString =>MyActualProcessingMethod(someString));

//Simulate a lot of messages
for(int i=0;i<100000;i++)
{
    myBlock.Post(someMessage);
}

When the messages finish and/or we don't want any more messages, we command it to complete, by refusing any new messages and processing anything left in the input buffer:

myBlock.Complete();

Before we finish, we need to actually await for the block to finish processing the leftovers:

await myBlock.Completion;

All Dataflow blocks can accept messages from multiple clients.

Blocks can be combined as well. The output of one block can feed another. The TransformBlock accepts a function that transforms an input into an output.

Typically each block uses tasks from the thread pool. By default one block processes only one message at a time. Different blocks run on different tasks or even different TaskSchedulers. This way, you can have one block do some heavy processing and push a result to another block that updates the UI.

string MyActualProcessingMethod(string it)
{
  // Process the string and do some other stuff
  // and send a progress message downstream
  return SomeProgressMessage;
}

void UpdateTheUI(string msg)
{
  statusBar1.Text = msg;
}

var myProcessingBlock = new TransformBlock<string,string>(msg =>MyActualProcessingMethod(msg));

The UI will be updated by another block that runs on the UI thread. This is expressed through the ExecutionDataflowBlockOptions :

var runOnUI=new ExecutionDataflowBlockOptions { 
                  TaskScheduler = TaskScheduler.FromCurrentSynchronizationContext() 
            };
var myUpdater = new ActionBlock<string>(msg => UpdateTheUI(msg),runOnUI);

//Pass progress messages from the processor to the updater
myProcessingBlock.LinkTo(myUpdater,new DataflowLinkOptions { PropagateCompletion = true });

The code that posts messages to the pipeline's first block doesn't change :

//Simulate a lot of messages
for(int i=0;i<100000;i++)
{
    myProcessingBlock.Post(someMessage);
}


//We are finished, tell the block to process any leftover messages
myProcessingBlock.Complete();

In this case, as soon as the procesor completes it will notify the next block in the pipeline to complete. We need to wait for that final block to complete as well

//Wait for the block to finish
await myUpdater.Completion;

How about making the first block work in parallel? We can specify that up to eg 10 tasks will be used to process input messages through its execution options :

var dopOptions = new ExecutionDataflowBlockOptions {MaxDegreeOfParallelism = 10};
var myProcessingBlock = new TransformBlock<string,string>(msg =>MyActualProcessingMethod(msg),dopOptions);

The processor will process up to 10 messages in parallel but the updater will still process them one by one, in the UI thread.

I will look into it. Thanks! – Ramesh Durai Sep 14 '17 at 13:51 — Ramesh Durai, Sep 14 '17 at 13:51

score 0 · Answer 2 · answered Sep 14 '17 at 13:22

You're best bet is to use a profile to monitor the running application and determine for sure where the CPU is spending it's time.

However, it looks like you have the possibility for a busy-wait loop if this.messageQueue.Count is 0. At minimum, I would suggest adding a small pause if the queue is empty to allow a message to go onto the queue. Otherwise your CPU is just spending time checking the queue over and over and over.

If the time is spent dequeueing messages, you may want to consider handling multiple messages at once (if there are multiple messages available), assuming you're queue allows you to pop multiple messages off the queue in a single call.

Reduce CPU usage while processing large amount of data

2 Answers2