Using TEvent and MsgWaitForMultipleObjects in blocking main thread

Question

I have found this Remy's interesting code. Delphi : How to create and use Thread locally?

Can this be done so I can do multiple threads and wait until they are all finished and then continue with main thread? I tried it like this but no success...

procedure Requery(DataList: TStringList);
var
  Event: TEvent;
  H: THandle;
  OpResult: array of Boolean;
  i: Integer;
begin
  Event := TEvent.Create;
  try
    SetLength(OpResult, DataList.Count); 
    for i:=0 to DataList.Count-1 do begin
      TThread.CreateAnonymousThread(
        procedure
        begin
          try
            // run query in thread
            OpResult[i]:=IsMyValueOK(DataList.Strings[i]);
          finally
            Event.SetEvent;
          end;
        end
      ).Start;
      H := Event.Handle;
    end;
    while MsgWaitForMultipleObjects(1, H, False, INFINITE, QS_ALLINPUT) = (WAIT_OBJECT_0+1) do Application.ProcessMessages;
    
    for i:=Low(OpResult) to High(OpResult) do begin
      Memo1.Lines.Add('Value is: ' + BoolToStr(OpResult[i], True));
    end;
  finally
    Event.Free;
  end;

  // Do next jobs with query
  ...
end;

You need an array of handles to wait on and a new TEvent for each thread. Now you only have one TEvent and the first thread that finishes will signal it. — J..., Oct 07 '20 at 19:56
You should not block the main thread by waiting for several other threads to finish and then do other things after that. Better put the worker thread creation and waiting for them in a thread, too! — Delphi Coder, Oct 07 '20 at 21:58
_"I tried it like this but no success..."_ - How come? What behavior did you get and what did you expect? — GolezTrol, Oct 07 '20 at 21:58

Remy Lebeau · Accepted Answer · 2020-10-08T14:40:42.730

Can this be done so I can do multiple threads and wait until they are all finished

Yes. You simply need to create multiple TEvent objects, one for each TThread, and then store all of their Handles in an array to pass to MsgWaitForMultipleObjects():

procedure Requery(DataList: TStringList);
var
  Events: array of TEvent;
  H: array of THandle;
  OpResult: array of Boolean;
  i: Integer;
  Ret, Count: DWORD;

  // moved into a helper function so that the anonymous procedure
  // can capture the correct Index...
  procedure StartThread(Index: integer);
  begin
    Events[Index] := TEvent.Create;
    TThread.CreateAnonymousThread(
      procedure
      begin
        try
          // run query in thread
          OpResult[Index] := IsMyValueOK(DataList.Strings[Index]);
        finally
          Events[Index].SetEvent;
        end;
      end
    ).Start;
    H[Index] := Events[Index].Handle;
  end;

begin
  if DataList.Count > 0 then
  begin
    SetLength(Events, DataList.Count);
    SetLength(H, DataList.Count);
    SetLength(OpResult, DataList.Count);

    try
      for i := 0 to DataList.Count-1 do begin
        StartThread(i);
      end;

      Count := Length(H);
      repeat
        Ret := MsgWaitForMultipleObjects(Count, H[0], False, INFINITE, QS_ALLINPUT);
        if Ret = WAIT_FAILED then RaiseLastOSError;
        if Ret = (WAIT_OBJECT_0+Count) then
        begin
          Application.ProcessMessages;
          Continue;
        end;
        for i := Integer(Ret-WAIT_OBJECT_0)+1 to High(H) do begin
          H[i-1] := H[i];
        end;
        Dec(Count);
      until Count = 0;

      for i := Low(OpResult) to High(OpResult) do begin
        Memo1.Lines.Add('Value is: ' + BoolToStr(OpResult[i], True));
      end;
    finally
      for i := Low(Events) to High(Events) do begin
        Events[i].Free;
      end;
    end;
  end;

  // Do next jobs with query
  ...
end;

That being said, you could alternatively get rid of the TEvent objects and wait on the TThread.Handles instead. A thread's Handle is signaled for a wait operation when the thread is fully terminated. The only gotcha is that TThread.CreateAnonymousThread() creates a TThread whose FreeOnTerminate property is True, so you will have to turn that off manually:

procedure Requery(DataList: TStringList);
var
  Threads: array of TThread;
  H: array of THandle;
  OpResult: array of Boolean;
  i: Integer;
  Ret, Count: DWORD;

  // moved into a helper function so that the anonymous procedure
  // can capture the correct Index...
  procedure StartThread(Index: integer);
  begin
    Threads[Index] := TThread.CreateAnonymousThread(
      procedure
      begin
        // run query in thread
        OpResult[Index] := IsMyValueOK(DataList.Strings[Index]);
      end
    );
    Threads[Index].FreeOnTerminate := False;
    H[Index] := Threads[Index].Handle;
    Threads[Index].Start;
  end;

begin
  try
    SetLength(Threads, DataList.Count);
    SetLength(H, DataList.Count);
    SetLength(OpResult, DataList.Count);

    for i := 0 to DataList.Count-1 do begin
      StartThread(i);
    end;

    Count := Length(H);
    repeat
      Ret := MsgWaitForMultipleObjects(Count, H[0], False, INFINITE, QS_ALLINPUT);
      if Ret = WAIT_FAILED then RaiseLastOSError;
      if Ret = (WAIT_OBJECT_0+Count) then
      begin
        Application.ProcessMessages;
        Continue;
      end;
      for i := Integer(Ret-WAIT_OBJECT_0)+1 to High(H) do begin
        H[i-1] := H[i];
      end;
      Dec(Count);
    until Count = 0;

    for i := Low(OpResult) to High(OpResult) do begin
      Memo1.Lines.Add('Value is: ' + BoolToStr(OpResult[i], True));
    end;
  finally
    for i := Low(Threads) to High(Threads) do begin
      Threads[i].Free;
    end;
  end;

  // Do next jobs with query
  ...
end;

Either way, note that MsgWaitForMultipleObjects() is limited to waiting on a maximum of 63 (MAXIMUM_WAIT_OBJECTS[64] - 1) handles at a time. The WaitForMultipleObjects() documentation explains how to work around that limitation, if you need to:

To wait on more than MAXIMUM_WAIT_OBJECTS handles, use one of the following methods:

Create a thread to wait on MAXIMUM_WAIT_OBJECTS handles, then wait on that thread plus the other handles. Use this technique to break the handles into groups of MAXIMUM_WAIT_OBJECTS.

Call RegisterWaitForSingleObject to wait on each handle. A wait thread from the thread pool waits on MAXIMUM_WAIT_OBJECTS registered objects and assigns a worker thread after the object is signaled or the time-out interval expires.

Or, you could simply process your list in smaller batches, say no more than 50-60 items at a time.

@DelphiCoder 63, actually. And this is clearly stated in the `MsgWaitForSingleObject()` documentation. The [`WaitForMultipleObjects()`](https://learn.microsoft.com/en-us/windows/win32/api/synchapi/nf-synchapi-waitformultipleobjects) documentation explains how to work around this limitation. I have updated my answer. — Remy Lebeau, Oct 07 '20 at 23:08
Another option would be to start a manager thread and have it just wait on all the child threads and then `Queue` whatever continuation from there. — J..., Oct 08 '20 at 00:58
You could also use the [.SyncObjs.TCountdownEvent](http://docwiki.embarcadero.com/Libraries/en/System.SyncObjs.TCountdownEvent) instead of all TEvent's. — LU RD, Oct 08 '20 at 04:55
@Remy. Great, Compiler complains that line "if (Ret >= WAIT_OBJECT_0) and (Ret < (WAIT_OBJECT_0+Count)) then" always evaluates to true. Also is it ok to send message to Memo1 from function IsMyValueOK with Synchronize? — davornik, Oct 08 '20 at 06:59
@davornik fixed. As for `Synchronize`, you can use it inside the anonymous threads, yes. It will get processed by the `Application. ProcessMessages` inside the wait loop — Remy Lebeau, Oct 08 '20 at 14:41

Using TEvent and MsgWaitForMultipleObjects in blocking main thread

1 Answers1