WCF One-Way Callbacks Timing Out?

Question

How is this possible? I thought one way calls were fire and forget. The method is marked as one-way. The callback concurrency mode is set to Multiple and the UseSychronizationContext of the callback class is set to false. The data being sent is not more than 1KB yet every time I send about 30-40 small messages concurrently, the calls start to block and eventually some of them timeout. I've benchmarked my client->server calls at about 16000/sec. When I try to call back to client, I can only muster about 2 per second, and this on a OneWay call!

My binding configuration for the server looks like so:

<system.serviceModel>
<bindings>
  <netNamedPipeBinding>
    <binding name="netNamedPipeBinding1" receiveTimeout="23:00:00" maxReceivedMessageSize="1048576" maxBufferPoolSize="1048576" maxConnections="500">
      <readerQuotas maxStringContentLength="99999999" maxArrayLength="9999999" maxBytesPerRead="999999"/>
      <security mode="None"/>
    </binding>
  </netNamedPipeBinding>
</bindings>
<behaviors>
  <serviceBehaviors>
    <behavior name="highThroughPut">
      <serviceThrottling maxConcurrentCalls="3000" maxConcurrentInstances="3000" maxConcurrentSessions="3000"/>
    </behavior>
  </serviceBehaviors>
</behaviors>
<services>
  <service name="OLII.Apps.Services.Data.DataServices.DataService" behaviorConfiguration="highThroughPut">
    <endpoint bindingConfiguration="netNamedPipeBinding1" address="net.pipe://localhost/DataListener" binding="netNamedPipeBinding" contract="OLLI.Apps.Services.ProxyClients.DataServerProxyClient.IDataListenerService"/>
   </service>
</services>
</system.serviceModel>

My callback contract looks like so:

   public interface IDataCallbackClient
    {
        [OperationContract(IsOneWay = true)]
        void GetData(string file, int id);
    }

My client callback class looks like so:

   [CallbackBehavior(ConcurrencyMode = ConcurrencyMode.Multiple, UseSynchronizationContext = false)]
    public class DataCallback : IDataCallbackClient
    {
public void GetData(string file, int id)
{
//If I put Thread.Sleep(5000);  When the server calls this method, the first few go through, and subsequent calls block.  If I do a return statement here, all the calls go through really fast on the server side.
//Does some processing with file and id.  It then goes back to server with data.
}
}

Perhaps throttling is used for callback as well? What version of .NET framework are you using? — Ladislav Mrnka, Apr 13 '11 at 18:02
I'm using .net 4.0. I noticed that on the first batch of concurrent calls it takes a very long time and sometimes the connection is even lost. The next batch of concurrent calls, it goes fast. Is there some JIT compilation issue here? How do I even code for this? I have throttling set very high: — Kayode Leonard, Apr 13 '11 at 18:18
What is even stranger is that if my callback method simply returns the callback is fast. However if I put a sleep in my callback method, the callback call on the server hangs as if its waiting for the client. However these are OneWay calls so why would the server wait for the client callback to return? — Kayode Leonard, Apr 13 '11 at 21:03
That sounds really strange. Can you post more information about your service and callback contracts and about binding? — Ladislav Mrnka, Apr 13 '11 at 21:19
Are you able to measure how many calls can client process before other calls are blocked and waiting? Is this always the same number? — Ladislav Mrnka, Apr 14 '11 at 18:53

score 2 · Accepted Answer · answered Apr 14 '11 at 19:07

I figured it out. I had calls to my service that were blocking and starving the thread pool in the process. I was also invoking calls on the thread pool from inside my wcf service, which is a bad practice since those methods are invoked on the thread pool themselves. Looks like when you make a one way call, and the thread pool is starved, the one way call will timeout since it doesn't have a thread to execute on. Thanks

WCF One-Way Callbacks Timing Out?

1 Answers1