4

Ok I am at my wit's end with this thing. I have a WCF duplex service. Here is how the architecture works:

  1. The client opens a connection to the endpoint and supplies a callback implementation
  2. The service takes that request and does some stuff on other threads(could be 1 second could be 2 minutes, which is the reason I am not using async operations)
  3. When the processing is done, it calls the callback of the client

The problem is when the service calls that callback, seemingly nothing happens. No error, no anything. Upon further investigation I found an exception in the server trace:

The I/O operation has been aborted because of either a thread exit or an application request

This happens right after attempting to execute the callback.

The client never receives a response, or closes. All that happens is, since the initial request is made on a thread different from the main one, it just waits there forever for that thread to complete.

The most weird thing is, that if I try to call the callback in the operation that the client invokes, without going into another thread, everything works fine - the callback is successfully called, which leads me to believe that I have configured the service correctly, but have a threading/deadlocking problem.

Here is how I am calling the service:

SubmissionServiceClient client = CreateClientInstance();
        client.Open();
        Guid executionId = await client.SubmitAsync(submission);
        submissionCompletionSource.Task.Wait(); //waits for the callback to be called (I omitted the extra wiring code for better readability)
        client.Close();

private SubmissionServiceClient CreateClientInstance()
{
    NetHttpBinding binding = new NetHttpBinding();
    binding.WebSocketSettings.TransportUsage = WebSocketTransportUsage.Always;
    EndpointAddress endpointAddress = new EndpointAddress("ws://localhost:9080/SubmissionRouter");
    InstanceContext instanceContext = new InstanceContext(this);
    SubmissionServiceClient submissionServiceClient = new SubmissionServiceClient(instanceContext,binding,endpointAddress);
    return submissionServiceClient;
}

This is the callback operation:

public void SubmissionProcessed(SubmissionResultDto result)
        {
            submissionCompletionSource.TrySetResult(result);
        }

This is the service operation that the client calls:

public Guid Submit(SubmissionDto submission, ISubmissionCallback callback)
        {
            ExecutionDto execution = new ExecutionDto()
            {
                Id = Guid.NewGuid(),
                Submission = submission
            };
            RequestExecution(execution); //Queues the execution of the operation
            submissions.Add(execution.Id, callback);

            return execution.Id;
        }

And this is where the service calls the callback of the client(this method is executed on a different thread from the one that the initial request was made on):

                ISubmissionCallback callback = submissions[submissionResult.ExecutionId];
                    callback.SubmissionProcessed(submissionResult);

Service behaviour:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, ConcurrencyMode = ConcurrencyMode.Reentrant)]

As you can see upon submission, the service stores the callback in a dictionary, paired with an id, which it later uses to retrieve the callback and invoke it. I believe the reason it is failing is because I am trying to do that on a different thread.

EDIT: I added a Ping operation to the service, that calls another thread, which hangs for 3 seconds and then calls a Pong function on the client.

public void Ping()
        {
            var callback = OperationContext.Current.GetCallbackChannel<ISubmissionCallback>();
            Task.Run(() =>
            {
                System.Threading.Thread.Sleep(3000);
                callback.Pong();
            });
        }

client: class Program { static void Main(string[] args) { NetHttpBinding binding = new NetHttpBinding(); binding.WebSocketSettings.TransportUsage = WebSocketTransportUsage.Always; EndpointAddress endpointAddress = new EndpointAddress("ws://localhost:9080/SubmissionRouter"); InstanceContext instanceContext = new InstanceContext(new Callback()); SubmissionServiceClient submissionServiceClient = new SubmissionServiceClient(instanceContext, binding, endpointAddress);

        submissionServiceClient.Ping();

        Console.Read();
    }

    public void SubmissionProcessed(SubmissionResultDto result)
    {
        throw new NotImplementedException();
    }
    class Callback : ISubmissionServiceCallback
    {
        public void Pong()
        {
            Console.WriteLine("Pong!");
        }

        public void SubmissionProcessed(SubmissionResultDto result)
        {

        }
    }
}

This actually worked successfully. I managed to receive my answer on the client side. I am now officially completely lost.

Phoenix
  • 913
  • 1
  • 8
  • 18
  • as a test could you try calling back on your thread a very basic method, eg one that does not take any args like `Ping` and see what happens? try to limit any other traffic/action on the channel after the connection is established so you can simplify the recreate. – wal Nov 09 '15 at 04:34
  • do you maintain a reference to the server-side connection? (not just the callback refs) – wal Nov 09 '15 at 04:50
  • Are you blocking the UI thread with `submissionCompletionSource.Task.Wait();`? Also why do you call `Wait` when you have `await` the line before, wouldn't you rather do `await submissionCompletionSource.Task`? – Scott Chamberlain Nov 09 '15 at 05:09
  • @wal I updated the post. I am not sure if that is what you wanted me to try, but it did give some interesting results. Also I am not sure what you mean by your second question. If you are asking if I am keeping a reference to the channel that I initially make the request from (SubmissionServiceClient), then the answer is yes, since the method never returns until the callback is called. submissionCompletionSource.Task.Wait(); <-- this line ensures that. I hope I explained it clearly. – Phoenix Nov 09 '15 at 05:10
  • @ScottChamberlain Yes, that line is meant to block the current thread, until I get a response. The await on the line before really doesn't matter much, as the Submit operation returns almost instantly(all it does is queue the request for execution and generate a Guid). It could very well be a synchrnous operation. Is there a difference between Task.Wait and await Task, and if there is, could you explain it to me? – Phoenix Nov 09 '15 at 05:17

1 Answers1

3

If you are blocking the UI thread with submissionCompletionSource.Task.Wait(); that is causing your deadlock. WCF Callbacks happen on the UI thread by default, you can change the behavior by using CallbackBehaviorAttribute

[CallbackBehaviorAttribute(UseSynchronizationContext=false)]
class Callback : ISubmissionServiceCallback
{
    public void Pong()
    {
        Console.WriteLine("Pong!");
    }

    public void SubmissionProcessed(SubmissionResultDto result)
    {

    }
}

or by not blocking the UI thread.

SubmissionServiceClient client = CreateClientInstance();
client.Open();
Guid executionId = await client.SubmitAsync(submission);
await submissionCompletionSource.Task; //awaits for the callback to be called without blocking the UI.
client.Close();
Scott Chamberlain
  • 124,994
  • 33
  • 282
  • 431
  • I already tried doing UseSynchronizationContext=false and that didn't help. I will try your second suggestion shortly. – Phoenix Nov 09 '15 at 05:18
  • You sir are awesome! Suggestion number 2 worked. Now if I could only understand why. – Phoenix Nov 09 '15 at 05:24
  • @Phoenix it happens because you synchronously waiting for an async operation, [here is a in-depth blog about the issue](http://blog.stephencleary.com/2012/07/dont-block-on-async-code.html) – Scott Chamberlain Nov 09 '15 at 17:01