Running async socket operations in order

Question

What I need is a class that does async operations in order.

class FooSocket
{
    private Socket _Socket;

    // Message is a class that wraps a byte array.
    public Task<Message> Receive() { /*Bla...*/ };
    public Task<int> Send(Message message) { /*Bla...*/ };
}

If I call Send, Receive and Send in this order, I need to send first and queue the remaining receiving and sending operations until the first receiving is finished.

I tried creating a main task field in the class and follow a MainTask = MainTask.ContinueWith(...) kind of approach. I even wrote a class called Sequencer which does exactly this but it somehow felt wrong, creating nested tasks (using Task.Factory.FromAsync methods) in continuations and stuff.

I also tried to make something like queuing TaskCompletionSource objects and returning their tasks in my Receive/Send methods, checking the queue in an infinite loop on a seperate thread but since I'll have about 200k of FooSocket instances, a thread for each also felt unwise. If I make it a thread-pool, I come to this "a thread-pool shouldn't be used for long-running operations" rule.

I feel close but can't be sure what is the most efficient way to order these jobs.

if you can use C#5 or the async extension or even better F# it's easy if not then you should IMHO go the Task.FromAsync/Task.ContinueWith way — Random Dev, Jul 17 '12 at 09:27
@Jodrell, kind of; all FooSocket instances should work synchronously when it comes to their own operations while acting as asynchronous in the caller's perspective. — Şafak Gür, Jul 17 '12 at 09:33
you can add async support for C#4: http://www.microsoft.com/en-us/download/details.aspx?id=9983 - but on your own peril (works for me but it's CTP AFAIK) — Random Dev, Jul 17 '12 at 09:37
@Carsten, I don't think the folks here would let me install it to our server but I may have a chance to convince them. How easy it would make it? — Şafak Gür, Jul 17 '12 at 09:42
@d4wn: more or less trivial - you basically just have to add some async/await keywords - you can find lot's examples and infos here: http://msdn.microsoft.com/en-us/vstudio/async.aspx — Random Dev, Jul 17 '12 at 09:46
check this out http://www.codeproject.com/Questions/767211/sockets-clientplusserver-with-await-async-csharp — Jaider, Sep 14 '14 at 04:27

score 2 · Accepted Answer · answered Jul 17 '12 at 12:54

2

I would use TPL Dataflow. You can either install it via NuGet or as part of the Async CTP. TPL Dataflow provides a basic BufferBlock<T> type which sounds like just what you need.

If you're just modeling a socket, then complete the Send tasks when the data is buffered to go out, continuously read into another buffer, and complete the Receive tasks when you read from that buffer. (Note: a "send" operation on a Socket completes when the data is buffered to the OS, not when it goes out on the wire or reaches its destination).

If you're modeling a higher-level command/response, then you should have one task that represents the entire command/response, as James suggested. I would do both; layering one on top of the other is easy with async/await support. (Note: a "receive" operation on a Socket may complete with a partial receive, so you need message framing).

Your architecture may need some work as well:

I'll have about 200k of FooSocket instances

That would definitely be a problem (I'm assuming you're using TCP/IP). There are only ~65K TCP/IP ports of which only ~16K are ephemeral by default, and you have to leave a significant amount of "breathing room" for the OS or it starts misbehaving. I would estimate only ~12K connections are realistic, unless you change the ephemeral range which could (in theory) get you up to ~59K. ~200K is just not possible - unless you change the ephemeral range and have multiple IP addresses with a load balancer.

answered Jul 17 '12 at 12:54

Stephen Cleary

437,863
77
675
810

About BufferBlock, [MSDN page of it](http://msdn.microsoft.com/en-us/library/hh160414(v=vs.110).aspx) doesn't have any information about usage and after finding [this](http://codingndesign.com/blog/?p=212), I still have a difficulty to understand how I can use it in my scenario. I also didn't quite get your second paragraph (sorry, new to sockets). I read the message farming post of yours, it is great but software of the devices I send data to are out of my control and are not capable of understanding prefixed lengths. I just send the command in one big chunk and read until I see . – Şafak Gür Jul 17 '12 at 20:28
As you stated it as impossible, I also need to be connected to ~200k of these devices to interact with them but I can use up to 5 servers to accomplish that. 10k is enough at the short term, anyway. After that I'll have enough time to look into saner ways like cloud or sufficient server farms... Don't know yet. – Şafak Gür Jul 17 '12 at 20:38
My [first link](http://www.microsoft.com/en-us/download/details.aspx?id=14782) is to an introductory TPL Dataflow document. And message framing works fine with delimiters; I just prefer length prefixing if I have a choice. – Stephen Cleary Jul 17 '12 at 20:55
Sorry, I guess I was sloppy and seeing "Download Center" made me think it is an installer. I'll definitely look into it in a few minutes. But like I said I have no control over the software of the devices (the receiving code) that I need to communicate with. So delimiters are no different than length prefixing in this particular example since our devices support neither, at least in their current versions. – Şafak Gür Jul 17 '12 at 21:10
Ah, I thought by you meant a delimiter. If you mean the actual end of the stream, then that would be very inefficient. But if you don't have a choice, then you don't have a choice. :) – Stephen Cleary Jul 17 '12 at 21:11

score 1 · Answer 2 · answered Jul 17 '12 at 12:06

If it's a request/response-type behavior, IMHO the task granularity should be a full round-trip instead of separating send and recieve.

You don't need the CTP for .net 4 support, you can use the async targeting pack.

What does the consuming code look like? A loop with await calls?

Running async socket operations in order

2 Answers2