1

I have been passing a few ideas around in my head about how to actually contain large amounts of connections using an IO type of architecture while maintaining KISS. Through examples on the web, it seems like most use a double/single linked list with CONTAINING_RECORD. And, as a newbie in IO servers ( though, improving every day), I too use a linked-list container for an IO architecture.

My question is, instead of using a single/double linked list for my connections, why cant I just build a large array and use CONTAINING_RECORD? Can I used STL vector? Would that work? Also, what are other type of containers that work best with a massive IO server.

Im in the process of re-writing the server architecture for my game server (after many revisions), and would like to head into the right direction this time around because id rather not have to rewrite it again in the near future.

Thank you for your time, and replies.

Edit: Currently my server architecture is (in a nutshell):

Main thread listening and accepting -> Pass over the socket into a container.
Worker threads(2-3) grab IO events for the container of sockets.
Worker threads Read/Write Data on that container.

Main thread and worker threads all use a linked-list. I want to get away from this.

User
  • 659
  • 2
  • 12
  • 29
  • If you are using something like `select` or `poll` to check status of connections, you might want to think about `std::unordered_map` as well, with the socket descriptor as key. And of course you can combine containers, e.g. use `std::map` for quick socket lookup with `std::shared_ptr` as data pointing to the actual structure in another container. – Some programmer dude Sep 11 '12 at 06:53
  • @JoachimPileborg - `select` or `poll` (or better yet, `epoll`, `kqueue`) is a good idea on Unix, or perhaps WinCE where there is no IOCP, but @Ohmages is right to focus on IOCP, it is the way to do scalable servers on Windows. – asveikau Sep 11 '12 at 07:28
  • @Ohmages - If you have more than one thread draining the IOCP I would stay away from vectors, reallocating could change the address which is painful to synchronize across threads. – asveikau Sep 11 '12 at 07:31

2 Answers2

1

Your "connection list" will probably have removals from any position, not just the end. For std::vector, removing elements in the middle is an O(N) operation, but for linked lists it can be O(1). (For single-linked lists this isn't trivial and may require an inconvenient API).

std::map may be an interesting choice as it offers both O(log N) finding and removing of elements.

MSalters
  • 173,980
  • 10
  • 155
  • 350
  • Would you suggest map, or unordered_map? – User Sep 11 '12 at 07:24
  • Wouldn't worry about that in the first iteration. It depends a bit on the exact I/O architecture. – MSalters Sep 11 '12 at 07:26
  • Would I be able to use the WSAOVERLAPPED (of the struct) as the has key for the map? – User Sep 11 '12 at 10:34
  • @Ohmages: Not directly. It doesn't have a suitable `operator<`. If you're NOT using completion routines, and you're using the `HANDLE` member of WSAOVERLAPPED, then you could write a predicate which sorts them by handle. – MSalters Sep 11 '12 at 10:45
  • When I try to assign the HANDLE of the WSAOVERLAPPED (hEvent) into a unordered_map the first two GetQueuedCompletionStatus hEvents work, but after that the hEvent is null. However, if I do use the WSAOVERLAPPED as the unique id, itll work each time. Am I doing something wrong? – User Sep 11 '12 at 15:00
  • @Ohmages: Might want to create a new question, if you can't get a small code sample to work.. – MSalters Sep 13 '12 at 08:45
0

As with all data structures, it depends very much on what you want to do with it.

In a previous job I spent most of my time working on a hugely multithreaded C++ server which, in its Windows incarnation, used IO Completion Ports (the Solaris backend used /dev/poll, which is not that dissimilar in several essentials). That one stored connection-related data structures in a map-like structure dating from before the STL, using the file descriptors as the key values. Thus whenever we got an event on a connection we could look up its related data structures by the descriptor the IO layer handed us. New connections were easy to handle - just add an entry to the dictionary - and closed connections could also be cleaned up quite trivially.

Naturally one has to be careful about cross-thread access to these structures and about operation ordering - since IO is inherently effectful, the ordering of operations is crucial. Fortunately IOCP won't give you another event on another thread for the same socket until you put the socket back into the CP, but the Solaris implementation had to also keep a structure linking file descriptors to worker threads in order to ensure that we only processed one event per socket at a time, and in strict order, and we also tried to inject subsequent events for a socket into the same thread to avoid having to potentially switch the socket's structures onto another processor which is a disaster for cache hit rates.

The basic summary though is that we found an appropriately-designed dictionary class to be incredibly useful for this sort of thing.

Matthew Walton
  • 9,809
  • 3
  • 27
  • 36
  • I am very interested in how the server worked, could you explain in more detail? The main server was an IOCP that received IOs then sent the IO data (im guessing the socket id?) to the Solaris, which looked up the data with key of the socket (using map)? If that is the case, how did you handle doing data manipulations on the Solaris? Did you just send the IO back information to send back to the client (and multiple clients if applicable?) – User Sep 11 '12 at 07:49
  • It was the same server - it just ran on both Windows and Solaris, so had two socket-level IO layers. So the server when running on Windows received data through the IOCP, looked that socket up in the map, retrieved the necessary data structures and used that to put together an object which could be pushed into a queue for processing by a thread pool. – Matthew Walton Sep 12 '12 at 07:44