4

I have EPOLLIN event for read data only. Is it ok to direct write the data without setting EPOLLOUT event?

Oktaheta
  • 606
  • 5
  • 21
  • 2
    Sure, it is not a *requirement*, but if the data can't be sent right away, the `write()` will either block the calling thread (blocking socket) or fail with `EAGAIN`/`EWOULDBLOCK` (non-blocking socket). In the latter case, you could just call `write()` again periodically, or you could use `EPOLLOUT` to know when the socket is ready to send data before calling `write()` again. – Remy Lebeau Aug 08 '18 at 23:57
  • Can I do `if (EWOULDBLOCK || EINTR || EAGAIN) then I retry to write` ? – Oktaheta Aug 09 '18 at 00:03
  • 2
    You can. But in such case, if the socket is blocked for awhile (network lag, unresponsive peer, etc), you would end up in a busy loop calling `write()` over and over, wasting CPU cycles. Better to put the calling thread to sleep waiting for `EPOLLOUT` (or have the thread go do something else for awhile and come back later) before calling `write()` again. – Remy Lebeau Aug 09 '18 at 00:08

1 Answers1

6

When using epoll with level triggered notifications, you will get constant EPOLLOUT notifications on a socket except for those brief periods of time that the output buffer for the socket is actually full. The impact of this is that your event loop will wake up even if you have nothing to send, and there was no data to receive either.

To deal with that, you have two practical choices:

  1. Disable EPOLLOUT until those times you actually get EWOULDBLOCK / EAGAIN. At that point, you enable EPOLLOUT, and disable it again once your send is complete.

  2. Enable edge triggered notifications, and leave EPOLLOUT enabled all the time. Now, you will only get EPOLLOUT notifications when the system changes from a output buffer full state to non-full state. However, EPOLLIN is now also also edge triggered, which means you only get one EPOLLIN notification until you drain the input buffer.

In comments, a third option was presented to you to implement your own polling loop to deal with the sockets with full output buffers. This can be made to work, but should not be necessary, since epoll can tell you when it is okay to write.

jxh
  • 69,070
  • 8
  • 110
  • 193
  • what if i do EPOLLIN for listenfd only, once accepted, bring along accepted fd and spawn new thread to do recv and send and close? It won't blocked the epoll, it also work alone in the thread. – Oktaheta Aug 11 '18 at 07:55
  • You never want to write to the accepting socket, so EPOLLIN only is fine for that socket. Generally speaking, pre-emptive thread per connection servers don't scale very well, but if the load is expected to be light, it works well and the code can be easier to understand. – jxh Aug 11 '18 at 08:01
  • I mean in psuedo coding fd = accept(...); new thread(fd) { recv(fd...); heavyProcess(); send(fd...); close(fd); } So I can handle multi threaded request concurrently? – Oktaheta Aug 11 '18 at 08:06
  • @Oktaheta My comment is the same. It doesn't scale, but if that doesn't matter, it works. – jxh Aug 15 '18 at 17:07
  • @jxh "Enable edge triggered notifications, and leave EPOLLOUT enabled all the time" should it not be "Enable edge triggered notifications, and start with EPOLLOUT disabled. When you get EWOULDBLOCK or EAGAIN on write enable EPOLLOUT" – asinix Sep 05 '19 at 10:16
  • 1
    @asinix: No. With edge triggered and EPOLLOUT always enabled, the application always attempts to write. When write blocks, it only waits for notification, since EPOLLOUT is already enabled. – jxh Sep 05 '19 at 16:35