-3

i have this part of code

bool hasData = true;
using (Context context = new Context())
{
    using (SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(MAX_THREADS))
    {
        while (hasData)
        {
            Message message = context.Database.SqlQuery<Message>($@"
select top(1) * from Message  where 
( Status = {(int)MessageStatusEnum.Pending} ) or 
( Status = {(int)MessageStatusEnum.Paused } and ResumeOn < GETUTCDATE() )
            ").FirstOrDefault();

            if message == null)
            {
                hasData = false;
            }
            else
            {

                concurrencySemaphore.Wait();

                tasks.Add(Task.Factory.StartNew(() =>
                {
                    Process(message);

                    concurrencySemaphore.Release();

                }, this.CancellationToken));

            }
        }
        Task.WaitAll(tasks.ToArray());
    }
}

And my Process function is something like this

private void Process(Message message)
{
    System.Threading.Thread.Sleep(10000);
}

Now, if i have only 1 item that i want to Process then the total execution time is 10sec and the execution time per item(1 item) is 10 sec. Well if i have 10 items for example, then the execution per item is increasing to 15-20sec.

I tried to change the value of MAX_THREADS but always if i have more than 10 items in my queue and start the parallel execution then the time of execution per item is about 15sec.

What i am missing?

Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
GomuGomuNoRocket
  • 771
  • 2
  • 11
  • 37
  • where is your "not parallel" code ? I doub't that `Process` is `Thread.Sleep` as you are doing some db acces in it (for not getting same message from `select top(1) * from Message`)... **what is the real question and where is the real code?** – Selvin Sep 24 '19 at 14:42
  • this is an example, that you can run by yourself. I am not looking for better code on Process fuction or on select query. I am just wondering, why if i have a lot of items running then the execution per item is slower than when i have only 1 item. – GomuGomuNoRocket Sep 24 '19 at 14:47
  • *this is an example, that you can run by yourself.* `context.Database.SqlQuery` <= no I can't I don't have Message class nor database with such table ... I bet that the problem is with database query not Tasks at all – Selvin Sep 24 '19 at 14:49
  • ok. i edited the sql query. now is more easy to understand the deeper question? – GomuGomuNoRocket Sep 24 '19 at 14:52
  • Use `Parallel.ForEach`, don't try to roll your own equivalent. And if you are trying to implement a queue across cpus or threads you need an atomic "get and remove from queue" operation not a select. And if it's not across CPUs/threads, why get just one at a time? – Ian Mercer Sep 24 '19 at 15:11
  • How many cores are there in your system? Tasks are not started immediately, but are queued for execution. If there are no free cores, they will wait. – Alexander Petrov Sep 24 '19 at 15:26

1 Answers1

0

Not unexpected. Parallel Slowdown is a very real problem. On the one end we have pleasingly parallel operations. On other end inherently serial stuff like calcuating the Fibonacci sequence. And there is a lot of area in between.

Multitasking in general and Multithreading in particular is not a magic bullet that makes everything faster. I like to say: Multitasking has to pick it's problems carefully. If you throw more tasks at a problem that is not designed for it, you end up with code that is:

  • more complex
  • more memory demanding
  • and most importanlty slower then the simple, sequential operation

At best you just added all that Task Management overhead to the operation. At worst you run into deadlocks and resource competition. And your example operation of Sleeping the Thread is not something that will benefit from Multitasking anyway. Please show us the actuall operation you want to speed up via Multitasking/-Threading. So we can tell you if there is any chance that can work out.

Kit
  • 20,354
  • 4
  • 60
  • 103
Christopher
  • 9,634
  • 2
  • 17
  • 31
  • this is an example, that you can run by yourself. I am not looking for better code on Process fuction or on select query. I am just wondering, why if i have a lot of items running then the execution per item is slower than when i have only 1 item. – GomuGomuNoRocket Sep 24 '19 at 14:47
  • @GomuGomuNoRocket: "At best you just added all that Task Management overhead to the operation." Every operation costs CPU cycles. This includes managing a bunch of task that run in a single thread, or managing a bunch of threads and swapping between them. You added that overhead without any speedup. How could it **not** take longer? – Christopher Sep 24 '19 at 15:55
  • So, because i added more tasks running parallel this behaviur is normal? Is normal the execution per item to be slower than seperate execution because 'i ask for c# to managent and schedule all this tasks?' . How can just add any speedup logic? – GomuGomuNoRocket Sep 25 '19 at 08:13
  • @GomuGomuNoRocket The only way to speed it up is to have **actuall code** for an **actuall problem** that would **actually benefit** from being split into tasks. This is not going to be something I can tell in any more general terms. – Christopher Sep 25 '19 at 13:12