1

This is a question regarding this answer: https://stackoverflow.com/a/14241095/2332808 (would comment it but newly created accounts apparently can't, sorry for the noise. Ressources on epollet/multithreading are hard to find...)

It suggests using epoll as the following:

  1. epoll_ctl() to activate notifications (and reactivate if EPOLLONESHOT is used).
  2. system input: read()/recv()/accept() in a loop until error EAGAIN.
  3. epoll_wait() to receive notifications.

But, assuming there are multiple threads in epoll_wait() on the same epollfd, wouldn't this risk having another thread being woken up on the same fd if it receives more data before you're done reading (e.g. ending up with two threads processing the same fd)

Even if you turn things around and read() till EAGAIN, epoll_ctl() and then call read() again to check there's still nothing (to avoid the race where you'd receive something between the last read and the epoll_ctl())...

BUT there is still no guarantee that you wouldn't have actually received something after the epoll_ctl() and would have both the last read() check and another thread woken up working on the same fd again...

I guess having a lock per fd would be an acceptable solution, but is that the "approved" use of epoll in edge-triggering mode with multiple threads polling on the same epollfd ?

Community
  • 1
  • 1
Asmadeus
  • 300
  • 2
  • 11
  • What are you trying to achieve with this complex machinery? How does `EPOLLONESHOT` help you here? – Nikolai Fetissov Apr 29 '13 at 19:48
  • I am having user callback when there is activity on some fd, so I'm trying to setup multiple threads to protect the user from itself (e.g. if one callback blocks). I was thinking `EPOLLONESHOT` would protect from having multiple worker threads being woken up on the same FD when there is a sudden activity spike on it. – Asmadeus Apr 30 '13 at 06:57
  • Well, from what I can tell it works and I have no functional problem, there's even a small performance increase with horrible callbacks (which all lock everything down) at the cost of more CPU usage. I guess having a separate pool of worker threads which would be woken up from a single epoll loop (so back to `EPOLLIN` without edges or oneshot) would be a better design, but transmitting information to the workers doesn't seem much easier to me. – Asmadeus Apr 30 '13 at 08:07

1 Answers1

0

Yes, you still need to do proper locking to guard against those cases you describe - and using a lock per fd is the most sensible approach to do that.

cmeerw
  • 7,176
  • 33
  • 27