Questions tagged [epoll]

epoll is a Linux 2.6 readiness notification API for sockets, pipes, and special event-, signal-, and timer descriptors which can operate both in level- and edge-triggered mode, although presently only level-triggered behaviour is in accordance with the documentation. As opposed to poll or select, epoll scales O(1) in respect to the number of descriptors and O(N) in respect realized events.

The epoll API is built around 3 functions:

  • epoll_create creates a new epoll instance and returns a file descriptor that refers to it. This descriptor can be operated on with the other epoll functions and can be added to a different epoll instance
  • epoll_ctl allows file descriptors (sockets, pipes, eventfd, timerfd, signalfd, and epoll) being added and removed to an epoll's set of monitored descriptors, as well as flags of existing descriptors being modified
  • epoll_wait will return up to maxevents queued events. If no events are available, it will return zero. If a timeout is provided and no events are available, epoll_wait will block for the duration of the timeout (a value of -1 means forever).

The conceptual idea behind the API is that applications usually have a certain set of descriptors that changes rarely if ever, but which needs to be observed for readiness many times. Also, typically a lot fewer descriptors are ready than open. epoll therefore separates copying the list of descriptors to watch from the actual watching and notifies registered listeners instead of iterating a list of descriptors.

The operation of level-triggered mode (default) is easy, since it is identical of how poll/select works. As long as the resource is ready (e.g. as long as there remains data to be read), every call to epoll_wait will return an event.

The operation of edge-triggered mode (EPOLLET flag) is more complicated, more error-prone, inconsistenly documented, and inconsistently implemented. In epoll(7), it is explained in terms of reading partial data causing the next call to epoll_wait to block until new data arrives, but not while some data remains in the buffers. It is therefore recommended to use non-blocking descriptors and reading until EAGAIN is received.
According to The Linux Programming Interface, edge-triggered mode only reports events that happened since the last call to epoll_wait.
In reality, it does a mixture of both (i.e. both reads and epoll_wait reset the status to "not ready"), and it does not work as indicated in respect of several epoll instances listening to the same socket or several threads waiting on the same epoll instance (observed under kernel 2.6.38 with timerfd and eventfd). Although epoll is supposed to signal all waiters upon arrival of an event, in edge-triggered mode it only ever signals a single waiter.

792 questions
13
votes
3 answers

Is level triggered or edge triggered more performant?

I am trying to figure out what is more performant, edge triggered or level triggered epoll. Mainly I am considering "performant" as: Ability to handle multiple connections without degredation. Ability to keep the uptmost speed per inbound…
Alex
  • 6,843
  • 10
  • 52
  • 71
13
votes
1 answer

What does epoll do with a file descriptor that refers to a directory?

Just like the title says, it I register a file descriptor that is a directory with epoll, what does it do?
Alex Gaynor
  • 14,353
  • 9
  • 63
  • 113
12
votes
2 answers

Open connections via Spring Websocket STOMP cause our server to die

So we use Spring websocket STOMP + RabbitMQ on the backend and we are having troubles with the open file descriptors. After a certain time, we hit the limit on the server and the server does not accept any connections, including both websockets and…
leventunver
  • 3,269
  • 7
  • 24
  • 39
12
votes
1 answer

When *exactly* is a socket ready to write?

When an application has a huge amount of data (400M) to write to a non-blocking socket, write() returns EWOULDBLOCK or EAGAIN when the send buffer becomes full. When the socket is (e)polled, I sometimes see a write-ready notification happening when…
themoondothshine
  • 2,983
  • 5
  • 24
  • 34
12
votes
1 answer

Does Nginx have separate queuing mechanism for requests?

Consider the following situation: you are deploying application that can serve 1 req./sec. What would happen if I send 10 request in 1 second? I wrote simple app to test that: https://github.com/amezhenin/nginx_slow_upstream . This test shows that…
Artem Mezhenin
  • 5,539
  • 6
  • 32
  • 51
11
votes
1 answer

What is correct way to use cProfile with asyncio code?

I'm trying to determine how to correctly use cProfile and pstats with asyncio code. I am profiling my script by running cProfile.run('loop.run_until_complete(main())', 'profile.stats'). After using pstats to sort by SortKeys.TIME, I get the…
J. Taylor
  • 4,567
  • 3
  • 35
  • 55
11
votes
3 answers

epoll IO with worker threads in C

I am writing a small server that will receive data from multiple sources and process this data. The sources and data received is significant, but no more than epoll should be able to handle quite well. However, all received data must be parsed and…
agnsaft
  • 1,791
  • 7
  • 30
  • 49
11
votes
1 answer

revisiting "how do you use aio and epoll together"

following the discussion at How do you use AIO and epoll together in a single event loop?. There are in fact 2 "aio" APIs in linux. There's POSIX aio (the aio_* family of functions), included in glibc and libaio developed I believe by RedHat (?),…
user237419
  • 8,829
  • 4
  • 31
  • 38
10
votes
1 answer

How to use boost::asio with Linux GPIOs

I have a single-threaded Linux application using boost::asio for asynchronous input/output. Now I need to extend this application to read in GPIO inputs on /sys/class/gpio/gpioXX/value. It is possible to do that with…
Florian
  • 1,036
  • 10
  • 15
10
votes
3 answers

epoll performance

Can anyone please help me to answer the questions about epoll_wait. Is it overkill to use many threads that call epoll_wait on the same fds set to serve at about 100K active sockets? or will it just be enough to create only 1 thread to perform…
Valentin
  • 860
  • 2
  • 10
  • 21
10
votes
1 answer

If a file is readable before epoll_ctl is called in edge-triggered mode, will a subsequent epoll_wait return immediately?

Does epoll guarantee that the first (or ongoing) call to epoll_wait after a file is registered with epoll_ctl for EPOLLIN and EPOLLET returns immediately in the case that the file was already readable prior to the epoll_ctl call? From my…
Andreas
  • 273
  • 1
  • 6
10
votes
2 answers

What's the advantage of using epoll_create1() instead of epoll_create()

I'm rewriting a multithread Linux-2.6.32+ application to replace select with epoll. The man pages for epoll_create1(2) declare that: If flags is 0, then, other than the fact that the obsolete size argument is dropped, epoll_create1() is the same as…
noisebleed
  • 1,299
  • 2
  • 12
  • 26
9
votes
3 answers

epoll, kqueue, /dev/poll .... extensions for PHP

Is there a PHP extension (stability is irrelevant) that allows for direct epoll, kqueue, /dev/poll polling functions without going through libevent or libev extensions?
Nick
  • 763
  • 1
  • 11
  • 26
9
votes
2 answers

Threading and scaling model for TCP server with epoll

I've read the C10K doc as well as many related papers on scaling up a socket server. All roads point to the following: Avoid the classic mistake of "thread per connection". Prefer epoll over select. Likewise, legacy async io mechanism in unix may…
selbie
  • 100,020
  • 15
  • 103
  • 173
9
votes
2 answers

Should I use epoll or just blocking recv in threads?

I'm trying to write a scalable custom web server. Here's what I have so far: The main loop and request interpreter are in Cython. The main loop accepts connections and assigns the sockets to one of the processes in the pool (has to be processes,…
Erin
  • 1,848
  • 15
  • 26