0

I'm going to use aio for async read. When aio completes and signal handler is triggered, I may need to do another aio_read call and proceed.

aio_read isn't mentioned among safe functions (in man signal). Ordinary read is, though.

What are the dangers of doing subsequent aio_read calls inside aio signal handler?

olegst
  • 1,209
  • 1
  • 13
  • 33
  • From [the `aio_read` manual page](http://man7.org/linux/man-pages/man3/aio_read.3.html): "Simultaneous I/O operations specifying the same `aiocb` structure produce undefined results." If the signal comes while doing another AIO call using the same control structure, that will lead to problems. If you use a separate control structure and separate buffers it should be safe, I *think*. I don't know if there will be problems with the AIO signaling if the request finished immediately. – Some programmer dude Mar 31 '16 at 09:10
  • Could you instead use a separate thread receiving the signals using e.g. [`sigwaitinfo()`](http://man7.org/linux/man-pages/man2/sigwaitinfo.2.html)? That way you could allocate a new control structure (or use mutexes) to avoid using a control structure already in use? – Nominal Animal Mar 31 '16 at 09:33

1 Answers1

3

As the author of proposed Boost.AFIO which can make use of POSIX AIO, I strongly recommend against using POSIX AIO at all. I am hardly alone in this opinion, @arvid is similarly against: http://blog.libtorrent.org/2012/10/asynchronous-disk-io/. The API itself is poorly designed and as a result scales poorly with load unless you use OS-specific alternatives or extensions to AIO like BSD kqueues. POSIX AIO is essentially useless as-is.

Additionally, AIO calls are not signal safe on Linux, which you are probably using. This is because on Linux they are implemented in userspace using an emulation based on a threadpool. On BSD, AIO calls have a proper kernel syscall interface, but in the kernel turn into - yes you guessed it - a threadpool based emulation unless O_DIRECT is turned on.

You are therefore much better off on POSIX of simply always using a threadpool unless all your i/o is with O_DIRECT on. If O_DIRECT is indeed always on, Linux provides a custom kernel API detailed at http://man7.org/linux/man-pages/man2/io_submit.2.html which is fairly effective, and on BSD if you replace signal driven handling with BSD kqueues (https://www.freebsd.org/cgi/man.cgi?kqueue, see EVFILT_AIO) then with O_DIRECT things can also scale well, better than a threadpool anyway.

Use of signal based completion handling on ANY POSIX platform has dreadful performance. AFIO v2 provides a generic POSIX AIO backend, and it is dreadful, dreadful, dreadful. Avoid like the plague.

Note that a threadpooled synchronous API design is portable, scales well for most use cases, and is what I (and indeed arvid) would recommend to anybody without highly specialised needs like writing a database backend where you need very tight control over the physical storage layer, and anything but O_DIRECT|O_SYNC isn't an option.

Ok, all that said, if you really really want to use signal driven aio, I assume this is because you want to multiplex your file i/o with non-file i/o stuff and you therefore can't use aio_suspend() which is the proper API for doing this. The way AFIO v2 handles this is to use a realtime signal to interrupt aio_suspend() when something not aio related needs to be processed, it can then be handled and aio_suspend() restarted. You need to be very careful in handling races and deadlocks, and you'll need to carefully mask and unmask the signal for the thread calling aio_suspend() lest the realtime signal gets lost and you get a lost wakeup. All in all, it's not worth it for the typically much lower i/o performance you get over a threadpool + synchronous APIs.

Niall Douglas
  • 9,212
  • 2
  • 44
  • 54