Why am I getting this result (same thread entering twice before exiting)?

Question

I've inherited a socket callback routine I'm trying to debug. It does asynchronous TCP communication with an EndReceive() immediately followed by a BeginReceve() so it's always listening. It's filled with "when hit" breakpoints that display a message and continue for diagnostics.

Because it's an asynchronous callback it may be called by the system multiple times for one BeginReceive() even if it's in the middle of processing a previous callback. Usually this is in a new thread, which is why I have a lock (a critical section) for the whole thing. But sometimes it seems to get reentered by the same thread even before it's exited. How is this possible? What am I doing wrong?

He's my trace (using 'When Hit' breakpoints) . . .

Socket_DataArrival() – Entering … managedThreadID=11
Socket_DataArrival() - first statement in lock . . . managedThreadID=11
Socket_DataArrival() - EndReceive() ... managedThreadID=11
Socket_DataArrival() - BeginReceive() . . . managedThreadID=11
Socket_DataArrival() – Entering … managedThreadID=11
Socket_DataArrival() - first statement in lock . . . managedThreadID=11
Socket_DataArrival() - EndReceive() ... managedThreadID=11
Socket_DataArrival() - BeginReceive() . . . managedThreadID=11
Socket_DataArrival() - exiting at end of routine . . . managedThreadID=11
Socket_DataArrival() - exiting at end of routine . . . managedThreadID=11

and here's the routine (proprietary code commented out) . . .

   private void Socket_DataArrival(IAsyncResult ar)
   {
       StateObject stateObject;
       int bytesReceived;
       int managedThreadId = Thread.CurrentThread.ManagedThreadId;

       lock (inputLock)
       {                 // "Entering..."
           try
           {
               _Receiving = false;  //"first statement in lock"
               stateObject = (StateObject)ar.AsyncState;
               bytesReceived = stateObject.sSocket.EndReceive(ar);    
               _Receiving = true;

               _StateObject = new StateObject(2048, _TCPConn);  //2048 = arbitrary number
               _StateObject.tag = "Socket_DataArrival ";

               _TCPConn.BeginReceive(
                    _StateObject.sBuffer,
                    0,
                    _StateObject.sBuffer.Length,
                    SocketFlags.None,
                    new AsyncCallback(Socket_DataArrival),
                    _StateObject);

            }
            catch (Exception exc)
            {
                subs.LogException("Socket_DataArrival", exc);
            }

            // proprietary stuff goes here

        }  // end critical section
        return;
    }

inputLock is defined at the class level as . . .

private Object inputLock = new Object();

How can the same thread enter it a second time before it's exited the first time?

have you tried to debug it, maybe put the whole stack trace in the log? — No Idea For Name, Nov 19 '14 at 15:35
It seems you enter recursion here, on the same thread - you have the same ID everywhere. Are you sure the Socket_DataArrival is called asynchronously? If you print the stack trace at the function entry, do you see the previous Socket_DataArrival call there? — galenus, Nov 19 '14 at 15:47

score 4 · Answer 1 · answered Nov 19 '14 at 17:05

It appears that when you call BeginReceive() and data is already available you get a callback immediately and on the thread that called BeginReceive().

This sounds completely plausible given that the underlying I/O model is almost guaranteed to be I/O completion port based and on operating systems after Windows XP (so all currently supported OSs) you can tell IOCPs to 'skip completion port processing on success' and instead return async data immediately to the caller.

So, what I assume is happening is that the BeginReceive() calls down into WSARecv() and this completes immediately because data is available and so the calling code executes the callback immediately on the calling thread. If there wasn't data available then the WSARecv() would return IO_PENDING and the I/O would eventually complete (when data arrived) and one of the threads that is associated with the socket's IOCP would then deal with the completion and call the handler.

Some data flow patterns are more likely to cause this to occur than others, and, of course, it's dependent on the network and how the data is flowing. It's also much more likely to happen with async sends...

The best way to fix it is to have your I/O handlers simply place all completions into a queue which only one thread can process and which can't be processed recursively. This then leaves you with a design whereby this problem can't happen. I wrote about such a design here, for the ACCU publication Overload.

score 2 · Answer 2 · edited Nov 19 '14 at 16:34

2

A lock only prevents a different thread from entering the critical section, not the same one.

Since you call _TCPConn.BeginReceive within Socket_DataArrival with same same callback (also Socket_DataArrival), this callback is simply entered again for when receiving data from _TCPConn.

Try to use a different callback for _TCPConn or try to call BeginReceive() outside of Socket_DataArrival().

edited Nov 19 '14 at 16:34

Kit

20,354
4
60
103

answered Nov 19 '14 at 15:57

helb

7,609
8
36
58

In this S.O. question [link](http://stackoverflow.com/questions/26806332/correct-use-of-beginreceive-endreceive#26806332) it says that _BeginReceive()_ should be done immediately after _EndReceive()_. You seem to be suggesting a different design pattern. What IS the correct .Net design pattern for asynchronours communication using _EndReceive()_ / _BeinReceive()_? – user316117 Nov 19 '14 at 17:04
1

That link doesn't have a code example, but I don't think they were talking about using semaphores or locks. A locked region of code should be kept as short as possible, and not have options for more asynchronous behavior within it. – Nerf Herder Nov 20 '14 at 19:23
I agree with @NerfHerder – helb Nov 21 '14 at 06:35

score 1 · Accepted Answer · answered Nov 19 '14 at 15:55

1

When you call BeginReceive(), you are allowing the next callback to happen (in that thread). Put the BeginReceive after the lock has been released.

answered Nov 19 '14 at 15:55

Nerf Herder

416
3
11

Why am I getting this result (same thread entering twice before exiting)?

3 Answers3