6

According to the documentation on aio_read/write, there are basically 2 ways that the AIO library can inform your application that an async file I/O operation has completed. Either 1) you can use a signal, 2) you can use a callback function

I think that callback functions are vastly preferable to signals, and would probably be much easier to integrate into higher-level multi-threaded libraries. Unfortunately, the documentation for this functionality is a mess to say the least. Some sources, such as the man page for the sigevent struct, indicate that you need to set the sigev_notify data member in the sigevent struct to SIGEV_CALLBACK and then provide a function handler. Presumably, the handler is invoked in the same thread. Other documentation indicates you need to set sigev_notify to SIGEV_THREAD, which will invoke the callback handler in a newly created thread.

In any case, on my Linux system (Ubuntu with a 2.6.28 kernel) SIGEV_CALLBACK doesn't seem to be defined anywhere, but SIGEV_THREAD works as advertised. Unfortunately, creating a new thread to invoke the callback handler seems really inefficient, especially if you need to invoke many handlers. It would be better to use an existing pool of threads, similar to the way most network I/O event demultiplexers work. Some versions of UNIX, such as QNX, include a SIGEV_SIGNAL_THREAD flag, which allows you to invoke handlers using a specified existing thread, but this doesn't seem to be available on Linux, nor does it seem to even be a part of the POSIX standard.

So, is it possible to use the POSIX AIO library in a way that invokes user handlers in a pre-allocated background thread/threadpool, rather than creating/destroying a new thread everytime a handler is invoked?

4 Answers4

2

I usually find it simpler and more portable to simulate asynchronous IO by doing ordinary IO in a dedicated background thread or threads, dispatching completion callbacks the way I like it.

Alexey Feldgendler
  • 1,792
  • 9
  • 17
1

One approach is to use a SIGEV_SIGNAL with a realtime signal to "carry" the ready file descriptor to a signal handler. Realtime signals queue and signal handlers execute asynchronously in one thread, so this approach is more-or-less functionally equivalent to SIGEV_CALLBACK:

/*
 * Warning!  Untested!
 * Also, safe initialization, per-thread signal masking and
 * error-checking omitted.
 */

static void my_callback(int sig, siginfo_t *info, void *context) {
  int fd;

  fd = info->si_value.sival_int;
  /* ...enqueue the fd for processing... */
}

struct sigaction sa;

sa.sa_handler = my_callback;                 /* Register our async callback */
sa.sa_flags = SA_SIGINFO;
sigaction(SIGRTMIN+1, &sa, NULL);
...

struct aiocb ac;

ac.aio_filedes = some_fd;
ac.aio_sigevent.sigev_notify = SIGEV_SIGNAL;
ac.aio_sigevent.sigev_signo  = SIGRTMIN+1;   /* Associate callback w. aiocb */
....
aio_read(&ac);

Now your my_callback will fire asynchronously in one thread, and it's up to you to pass the fd to your helper threadpool. See also this bit of SGI code demonstrating how to fall back to SIGEV_SIGNAL when SIGEV_CALLBACK isn't available.

pilcrow
  • 56,591
  • 13
  • 94
  • 135
0

This is a really old thread but it showed up at the top of Google when looking into this same problem. There is now a GNU extension aio_init which allows you to specify the maximum number of threads that aio should use and the lifespan of those threads.

Curtis Bezault
  • 53
  • 1
  • 7
0

If you are worried about several threads being created/destroyed for each completion call, why dont you batch your IOs ?

use list io apis..

struct aiocb **myaiocb;
 [just an array of aiocb pointers, where each aiocb points to an IO buffer
  and the operation to be performed, etc]
....
lio_listio(LIO_NOWAIT, myaiocb, num_ios, &sigevent);

The advantage with list IO is that the callback handler is called only after all the IOs in the list complete (successfully/unsuccessfully). You can check the status of each IO op using aio_return.

Tautology
  • 385
  • 2
  • 9