5

In deciding to implement asynchronous sockets in my simple server (linux), I have run into a problem. I was going to continually poll(), and do some cleanup and caching between calls. Now this seems wastefull, so I did more digging and found a way to possibly implement some callbacks on i/o.

Would I incur a performace penalty, and more importantly would it work, if I created a socket with O_NONBLOCK, use SIOCSPGRP ioctl() to send a SIGIO on i/o, and use sigaction() to define a callback function during i/o.

In addition, can I define different functions for different sockets?

Jon Weldon
  • 115
  • 2
  • 8
  • why don't you look at things like [`libevent`](http://libevent.org/) and [`libev`](https://github.com/brimworks/libev) ? – c00kiemon5ter May 01 '12 at 15:12
  • I'm trying to use as bare assembly as possible ^^ – Jon Weldon May 01 '12 at 15:15
  • I've once tried this (and succeeded!) but the resulting program spent 80% of its time tinkering with sigprocmask. The point is: you cannot call malloc() in the signal handler, so you'll have to preallocate with the SIGIO disabled. A plain select() or poll() loop (possibly with threads) might be easier to manage. For reference: IIRC NTP uses SIGIO. Mightbe DNS/bind does so too, dunno. – wildplasser May 01 '12 at 15:23

1 Answers1

3

"I was going to continually poll(), and do some cleanup and caching between calls. Now this seems wasteful"

Wasteful how? Did you actually try and implement this?

You have your fd list. You call poll or (better) epoll() with the list. When it triggers, you walk the fd list and deal with each one appropriately. You need to cache incoming and outgoing data, so each fd needs some kind of struct. When I've done this, I've used a hash table for the fd structs (generating a key from the fd), but you are probably fine, at least initially, just using a fixed length array and checking in case the OS issues you a weirdly high fd (nb, I have never seen that happen and I've squinted thru more logs than I can count). The structs hold pointers to incoming and outgoing buffers, perhaps a state variable, eg:

struct connection {
   int fd;  // mandatory for the hash table version
   unsigned char *dataOut;
   unsigned char *dataIn;
   int state;  // probably from an enum
};

struct connection connected[1000];  // your array, or...

...probably a linked list is actually best for the fd's, I had an unrelated requirement for the hash table.

Start there and refine stepwise. I think you are just trying to find an easy way out -- that you may pay for later by making other things harder ;) $0.02.

CodeClown42
  • 11,194
  • 1
  • 32
  • 67
  • The point of using callbacks is so I can devote more time to management of resorces, defragmenting memory (I'm also implementing my own malloc). I'm not sure when I should stop polling and start maintainance. I don't want to miss a connection because my server is busy trying to figure out what I can move to free up memory ;p I guess I'm just not sure how to work the scheduling, so implementing some sort of callback could provide a solution. Also, I'm trying to stay away from threads. (I've worked with them before, I just want to try to have it in one thread/process) – Jon Weldon May 01 '12 at 17:09
  • I doubt interrupting the mm routines via a callback to then deal with connections is going to make life easier. Write the routines so they do a little bit at a time, frequently. This fits with the async model. Set an appropriate timeout for poll when idle. When poll() returns, *at the end* of the fd list handling stuff, you check to see if it is time to run some quick mm stuff before you loop round and call poll() again. At that point, if the server is busy, you've just passed a bunch of stuff out to the OS socket layer for it to deal with; that's the best time to do your (quick bit of) mm. – CodeClown42 May 01 '12 at 18:03
  • You could also determine if the server is idle (or, come up with a metric for "how busy") based on what's up with the fd list when poll() returns, and use that to determine how much (if any) mm stuff is appropriate at the end of that cycle. Basically, I think you should think about how to integrate the mm with the socket handling rather than treating them as two completely unrelated and independent activities of a single process. Otherwise, thread. – CodeClown42 May 01 '12 at 18:07
  • "Wasteful how?" - well, I believe the preferred method is to let the thread sleep until there's work to be done. Polling just spins, so the thread is consuming CPU cycles. Those CPU cycles could be used for other [useful] work, hence its a waste to poll and spins. – jww Sep 13 '13 at 03:11
  • @noloader `poll()` doesn't "just spin" if there's nothing to do. It's a passive wait. A blocking call. Read the man page. You've lost the plot if you believe checking to see if there's mm to do every 30 seconds *while the server is idle anyway* represents "a waste of CPU cycles". It's about as cheap a method as you are going to get...and when the server isn't idle *yes, poll() would be spinning*, lol, how else are you going to manage a socket? – CodeClown42 Sep 13 '13 at 10:19
  • "A blocking call" -- regardless of whether the underlying sockets are NON_BLOCK or not. – CodeClown42 Sep 13 '13 at 10:25