Backgroundworking Thread Delphi 2010

Question

Can someone tell me how to create a thread in Delphi 2010 which will be "standby" for data to be calculated (not to terminate after its calculation task)?

I created a program which takes data from an external source via Indy UDPServer. The IdUDPServer1UDPRead event collects data and calls different threads (depends on the type of data), but while debugging the program, I saw that the thread is being terminated after the calculation and then is being created again (the creation of the thread is taking some time). Can I create fork threads of the same thread as long as the frequency of the incoming data is greater than the CPU (or thread) can handle (a data is coming before the thread finishes the calculation).

---

This is the code I am trying to write:

procedure TForm1.IdUDPServer1UDPRead(AThread: TIdUDPListenerThread;
    AData: TBytes;
    ABinding:TIdSocketHandle);                                             
begin   
  form1.panel2.color:=clLime;    
  ParseDelimited(IdUDPServer1.ReceiveString,'&');    
  if (Parsedelimited=1) and (Jvthread1.Terminated=true) then
    Jvthread1.Execute(self)    
  else if (Parsedelimited=2) and (Jvthread2.Terminated=true) then
    Jvthread2.Execute(self);

  Application.ProcessMessages;
  // i know this command is not very good but by removing this line the
  //gui is responding //after 2 or 3 sec
end;

The problem is that the times it takes for the JVThread1 or JVThread2 to do the calculations is greater than the incoming data and it is my belief that this problem is caused by the time needed to create again and again the thread (maybe this is a wrong speculation). Nevertheless this is partially solved by buffering the incoming UDP Data into Indy UDPServer but when I am trying to close UDPServer nothing happens until its buffer is totally empty, which takes approximately 3-4 seconds.

Answer 1

the creation of the thread is taking some time

True.

Can i create fork threads of the same thread as long as the frequency of the incoming data is greater

This does not make sense. You computer only can simultaneously execute as many threads as there is (number of CPUs) x (number of cores in one CPU). All other threads would be waiting for their timeslice. No matter how many thread you would create - there would only be few that can be executed. And creating more thread - only wastes system resources.

So the correct approach is I think not to terminate the thread, that calculated the data frame, but instead to request the next job load and to go into sleep if there is not immediate job ready.

Overall I suggest you too look at pipeline model: split processing your data into several stages.

(internet) -> input queue -> (N worker threads, some sleeping some active) -> queue of half-processed data -> (N worker threads, some sleeping some active) -> queue of fully processed data -> ( M results-saving threads )

N is limited by your total CPUs cores count, as I said above. M is limited by available end storage. Usually it is a single database on a single HDD, so M==1, but the storage might be sharded to different servers or different disks sometimes.

If your data is of different kinds you can fork the flow so that some stage would output the results into two different queues handled by different worker pools, judging by the kind of data.

The trick is to arrange an easy to use framework where threads would be easily put to sleep and automatically awakened as the input data goes. OmniThread library might be a possible solution for that, since you use Delphi 2010: http://otl.17slon.com/book/doku.php?id=book:highlevel:pipeline

---

Also after reading http://robertocschneiders.wordpress.com/2012/11/22/datasnap-analysis-based-on-speed-stability-tests/ it seems the Indy is not optimized for a many-connections applications. Since UDP is a sessionless protocol, i'd expect a new session be opened for any new connection and maybe would show similar performance limitation.

I suggest you to try to design a fake server on different platforms, and try to overload (DDoS) it to compare the maximum throughput different libraries would allow. Indy is one, then there are OverByte ICS and Ararat Synapse and there is Synopse mORMot. If you would design your program in a pipeline pattern, then I think you would be able to easily switch initial TCP/UDP input stage with any other library. But maybe you would be able to prove your library suits your application goal the best before writing the rest of the code? But then your initial stage is to be very simplistic - just receive the data and put it to queue without analyzing, let further stage be dispatchers looking what kind of data being received and sending it to different processing queues.

Answer 2

Since your question is understandably high level, I'm going to provide a high level answer which should point you in the right direction. If you then have more specific questions, please first check if someone else has already asked the question; if not then feel free to ask a new question.

Whenever you implement a thread, you have full control over what the thread does. So yes, if your thread execute method simply performs a calculation then ends, your thread will terminate. If you want to keep your thread active, just make sure the method doesn't end.

The easiest way to do this would be to use while True do;. WARNING Putting your thread into a simplistic infinite loop like this is actually very bad because it introduces some serious problems. You do need an infinite loop to keep the thread active for multiple calculations as they become available; however, you'll need additional code to resolve the following problems.

• A naive loop like that would simply hog a core of your processor (even if the loop does nothing). This is a waste of resources and can negatively impact other applications. What you really want is for the thread to only be processing while it actually has work to do; and be stopped/paused otherwise.
  • This can be done by using a Windows API call (or Delphi equivalent) that tells the thread to stop until something else happens.
  • Do some reading on the following: SleepEx, WaitForSingleObject, TSimpleEvent
• The second serious problem is that a loop like that cannot end gracefully. By ending gracefully, we mean the execute method has a way to exit normally after "cleaning up". Otherwise if you force it to exit in the middle of do something, it might have resources, locks or memory allocated that cannot be released.
  • So the first thing you'd want to change in your loop is that instead of while True, you should rather use while not Terminated. This way anything that has a reference to your thread can set a flag telling your thread it must exit its loop (thereby terminating) as soon as it's appropriate to do so.
  • Note that if your thread is paused (as per the earlier problem), you must also ensure that setting this flag wakes the thread up to actually check the flag. So you can add to your earlier reading list: WaitForMultipleObjects.
• The final problem you'll have to deal with, is getting new data to your thread so that it can perform the calculations. Usually when people create threads to perform one-off tasks, the required data is passed when the thread is created. The naive approach of simply allowing the main application to update the data is absolutely no good! You'd risk overwriting input data before the calculation is done; or worse, overwrite some inputs in the middle of a calculation creating inaccurate and unpredictable results.
  • Basically, you'll need to maintain a list of input data structures. Whenever you receive new data, you add a structure to the list. The thread will remove items from the list as it processes them. (This list would ordinarily be a Queue or FIFO collection.)
  • Whenever one or more items are added to the Queue, the wake-up event must be triggered so the thread can start processing again. The thread should process all items currently in the Queue before going back to sleep.
  • NB! It is absolutely essential that your Queue implementation be thread safe. You don't want concurrent adding and/or removing to corrupt your internal structures.
  • Suggested reading: Message Queues, PostMessage (sends a message to a standard Windows message queue) and Command design pattern.

---

Side Note:

You seemed concerned about the time overhead of creating a thread. This suggests your calculations are themselves actually quite fast. This begs the question, are you really gaining anything by using a separate thread to do your calculations?

You did also say that your input data already comes from different threads. So if your calculation involves correlating data from multiple source threads, then it does make sense. I only mentioned this because it's an all too common programming misconception that multiple threads can speed up a program. And if the calculations are fast there's absolutely no point in moving to another thread to do the same work.