0

For the past couple of days I have been thinking about how to solve one of my problems I am facing, and I have tried to research the topic but don't really know what I can do.

I have 2 sockets in the same struct that both have the same completionport. Problem is, they both use different protocols. Is there a way that I can find out which socket got triggered? Their called game_socket, and client_socket

Example code would be something like...

while (true) {
error = GetQueuedCompletionStatus(CompletionPort, &BytesTransfered, (PULONG_PTR)&Key, &lpOverlapped, 0);
srvc = CONTAINING_RECORD ( lpOverlapped, client , ol );
if ( error == TRUE ) {
cout << endl << "SOCKET: [" << srvc->client_socket << "] TRIGGERED - WORKER THREAD" << endl;
cout << endl << "BytesTransfered: [" << BytesTransfered << "]" << endl;

if ( srvc->game_client triggered ) {
// .. this code
} else {

// .. this code
}

Any ideas or help would be appreciated :)

User
  • 659
  • 2
  • 12
  • 29

2 Answers2

4

When using IOCP you have two items of user data that you can use per asynchronous operation.

The first is the completion key which is "per connection" data and is set when you associate the file handle or socket with the completion port by calling CreateIoCompletionPort() with the socket/file handle and an existing completion port. This value is returned on every call to GetQueuedCompletionStatus() for a given connection.

The second is the 'extended overlapped' structure which is "per operation" data. Each concurrent operation MUST have a unique overlapped structure assigned for it.

In your design above you should be using the "per connection" data to identify your connection. You have two connections (sockets) that are linked, so I would have two linked classes, one for each side of the connection. The class (or struct if you're using C rather than C++) would hold the socket, a pointer to the other half of the connection and a flag that indicates which type of connection it is. If using C++ these two classes would derive from a common base class so that you could call a member function to determine which type of connection the completion key represents. If using C use a discriminated union.

You then simply cast your completion key to the base, then determine which connection type you have and cast to the correct class/struct. You now know which connection the operation was performed on. The connection class can store all of the state for the state machine that runs its protocol.

As Criag mentions, each overlapped structure is "unique per operation" and in my IOCP code this tends to allow access to the data buffer for the operation and a flag that tells me what the operation was (read/write/accept/connect, etc).

One potential piece of complexity is keeping the per operation and per connection data alive until all outstanding operations that use it are complete. I personally favour reference counting of both the per connection and per operation data, but I'm sure there are other ways.

I have some example code for IOCP that uses these concepts (though doesn't provide a 'two connection' example), you can download it from here: http://www.serverframework.com/products---the-free-framework.html

Len Holgate
  • 21,282
  • 4
  • 45
  • 92
  • This is in my opinion a more precise answer than the accepted one (though that one is not wrong). The `CompletionKey` value is exactly what one could use in this case. – Damon Sep 05 '12 at 10:45
  • 1
    Damon; you need a separate OVERLAPPED per operation. In a simple system where you never have more than one read OR one write pending at a time then a single separate OVERLAPPED per socket would suffice; anything more complex needs a unique OVERLAPPED PER concurrent operation. I find that even simple transactional servers will tend to work better if the next recv is issue at the end of processing the current recv completion rather than after processing the corresponding write completion from the reply that you've written... – Len Holgate Sep 05 '12 at 11:42
  • Sorry, my wording was misleading (or indeed wrong). Yes, you certainly need to provide the memory location for a unique `OVERLAPPED` for each single request (say, sendto or readfile) that you fire off, but these are just "opaque" blocks without much useful information that the OS needs during the overlapped operation (I use them to piggyback some other data). In my above "for cosmetics, not because it's needed", I thought of the kind-of-rubbish ptr-to-`OVERLAPPED` that's filled in when getting a queued status, but on a second thought, even that won't work reliably with concurrent threads. – Damon Sep 05 '12 at 13:18
  • (deleted that line above, because it's really misleading) – Damon Sep 05 '12 at 13:19
2

The overlapped extension structure you're using is ideally suited for a specific connection instance. That structure, for example should not only contain the socket the operation is being performed on, but the action as well. For example, an IO completion on a socket that could be reading or writing should reflect which state was present at the time the IO request was started. Most examples of IOCP-based client/server code will attest to this general ideology.

In your case, you should NOT be using the same OVERLAPPED structure for two different sockets with two potentially independent actions. If the sockets are 'related' in some sense then that should be tracked and maintained outside of your OVERLAPPED structure. Each should have its own distinct structure for its own distinct IO completions. Anything else is honestly asking for a headache.

All that being said, it doesn't appear your use case can be far off from transforming your current scheme to accommodate what I'm describing. I'm always reluctant to give "you shouldn't do it that way" answers, as i totally hate hearing them myself, but it is none-the-less likely warranted in this case.

All the best.

WhozCraig
  • 65,258
  • 11
  • 75
  • 141
  • I was afraid of that. As I have tried numerous things like , WSAEventSelect with WSAEnumNetworkEvents , but nothing worked. :( – User Sep 03 '12 at 04:19
  • It may not be as bad as you think. You *can* use the same IOCP for multiple sockets without a problem. Just make sure each IO request is associated to one, and ONLY one, OVERLAPPED structure. If you have two sockets being managed in some object instance, and they're both potentially waiting on an IOCP, then each should likely have an extended OVERLAPPED structure that link to the object *and* identifies which socket is participating in the IO request represented by that OVERLAPPED data. I hope that made sense. You can do this, just think about it a little longer. – WhozCraig Sep 03 '12 at 05:27
  • I kinda figured I would have to-do something like that! Thanks for the reply , I will surely try this out with some small test servers. :) – User Sep 03 '12 at 16:32
  • Please note that the idea of using one `OVERLAPPED` with two sockets is conceptually wrong. In reality, you use one `OVERLAPPED` to retrieve _one event_, whatever that may be. This event may come from either one of two sockets, but that does not matter at all. It will have the associated `CompletionKey` which lets you tell what it refers to. – Damon Sep 05 '12 at 10:50