0

I read this question I/O Completion Ports *LAST* called callback, or: where it's safe to cleanup things

And i can not get my issue solved. The answer does not fully cover this method. I have also searched a lot here and in google but can not find a solution so i am opening a question here hope that is not duplicated.

In a multi-threaded IO Completion ports design when to increase the RefCount of the Per Socket structure? ie the CompletionKey. Currently i do increase it before calling WSARecv and if the return value of the call is not 0 or ERROR_IO_PENDING by last error, i decrease it and call a cleanup function, this function will check if the RefCount is 0, if it is then it will free the Per Socket structure. Else it will just free the Per IO structure (the one of OVERLAPPED), i also increase it before issuing any WSASend using the same way mentioned above. This RefCount is atomic using CRITCAL_SECTION. Upon returning from GetQueuedCompletionStatus i also decrease the RefCount.

However i have some questions about this method

I have a function that sends files from the main thread, the function is reading the file and issuing a PostQueuedCompletionStatus to do a send using WSASend through the IO Worker threads, the function sends file in chunks and when each chunk completes the IO Worker threads will inform the main thread with PostMessage to issue another send of the next chunk.

Now where i am supposed to increase this RefCount? in the main thread just before issuing a call to PostQueuedCompletionStatus? but what if a return from GetQueuedCompletionStatus returned and freed the Per Socket structure and the main thread still using it? (Example the main thread is executing the send function but not yet increased the RefCount) i tried to increase RefCount in the WSASend function in the IO Worker threads but it is the same issue

For instance: what if a thread woke up from GetQueuedCompletionStatus as a socket closure(caused by the outstanding WSARecv) and decrement the RefCount and it became 0 so it will free the Per Socket structure while a WSASend is executing in another IO Worker thread but not yet increased the RefCount? then obviously the thread that is about to issue a WSASend call will crash with access violation whenever it tries to enter the critical section.

Any idea how to synchronize access to this structure between IO Worker threads and the main thread?

LYC
  • 1
  • all is very simply - you need increment ref count to object, before pass it pointer to somebody. based on your description hard understand what you doing - need view code instead. however - *This RefCount is atomic using CRITCAL_SECTION* - you not need use `CRITCAL_SECTION` but interlocked increment/decrement. *to do a send using WSASend through the IO Worker threads* - for what ?! just do asynchronous send from current thread. you not need post this operation to another thread – RbMm May 06 '18 at 09:04
  • all your description only say that you have wrong design. with correct ref counting all work perfect – RbMm May 06 '18 at 09:07
  • Interlocked functions require a 32-bit align. What is wrong with a Critical section?, and why would i need to issue a WSASend from the main thread? that is what the IO worker threads are there for, the question is fairly simple: Once you decrement the refcount(WSASend) completion , issue another WSASend that will increase the RefCount, a disconnect might happen in between these 2 Dec/Inc and thus the ref will be 0 in that time causing the Per socket structure to be freed by a thread that returned from GQCS while another thread is in progress but not yet issued RefCount Increment. – LYC May 06 '18 at 10:30
  • critical section internal use interlocked functions which...require a 32-bit align. `CRITICAL_SECTION` - so also require align. even not 32, but 32 or 64 (on 64bit system). bit. in what problem have correct aligned `LONG _dwRef` ?! use critical section here - very not efficient. *and why would i need to issue a WSASend from the main thread?* - you not need, you **can**. and what you write next - say only that you complete not understand ref counting – RbMm May 06 '18 at 10:41
  • 1
    before increment ref counting on some structure - you must already have reference to it. so it never decrement to 0 at this moment. again - end of your comment only say about you not understand how use ref counting at all – RbMm May 06 '18 at 10:43
  • *might happen in between these 2 Dec/Inc and thus the ref will be 0 in that time causing the Per socket structure to be freed by a thread that returned from GQCS while another thread is in progress but not yet issued RefCount Increment.* - no. this never can happen, because before do RefCount Increment you must **already** have reference to data – RbMm May 06 '18 at 10:46
  • I never know that critical sections require align too. Have not seen it documented in MSDN, i use the heap functions to allocate the context required which as far as i know are not 32bit aligned, mind you direct me to some articles that explain ref counting more in depth? – LYC May 06 '18 at 10:48
  • are you @Wad ? under another nick ? how about think yourself ? **every** structure require some align - see `__alignof` of it. in what problem have 32 bit aligned long variable ? and again - when we acquire/release critical section - interlocked functions called on in data internal. you yourself say that align require for it – RbMm May 06 '18 at 10:52
  • Ok thanks for the info. And no i am not Wad. – LYC May 06 '18 at 10:58
  • every structure must be aligned. you not need critical section for interlocked increment/decrement. for increment ref count at some point - you **already** must have reference to this structure at this point - as result it never became invalid here. in this and sense of ref counting. initially, when you create structure - you have 1 as ref count. want pass it to another thread - inc ref count before this. want pass to to function (io) - inc ref count before this. save it pointer in another structure - inc ref count before this. etc – RbMm May 06 '18 at 11:03

0 Answers0