linux - Can't get eventfd to work with epoll together

Question

I'm writing a simple server class based on epoll. In order to wake up epoll_wait(), I decided to use an eventfd. It is said that it is better suited for simple event communication and I agree with that. So I created my event and put a watch on it:

_epollfd = epoll_create1(0);
if (_epollfd == -1) throw ServerError("epoll_create");
_eventfd = eventfd(0, EFD_NONBLOCK);
epoll_event evnt = {0};
evnt.data.fd = _eventfd;
evnt.events = _events;
if (epoll_ctl(_epollfd, EPOLL_CTL_ADD, _eventfd, &evnt) == -1)
    throw ServerError("epoll_ctl(add)");

later in the message waiting loop, on a separate thread:

    int count = epoll_wait(_epollfd, evnts, EVENTS, -1);
    if (count == -1)
    {
        if (errno != EINTR)
        {
            perror("epoll_wait");
            return;
        }
    }

    for (int i = 0; i < count; ++i)
    {
        epoll_event & e = evnts[i];
        if (e.data.fd == _serverSock)
            connectionAccepted();
        else if (e.data.fd == _eventfd)
        {
            eventfd_t val;
            eventfd_read(_eventfd, &val);
            return;
        }
    }

and, of course, the code that stop the server was:

eventfd_write(_eventfd, 1);

For reasons that I can't explain, I was unable to wake up the epoll_wait() just by writing to the event. Eventually, this worked in a few debugging sessions.

Here is my workaround: knowing that EPOLLOUT will trigger an event every time the fd is available for writing, I changed the stop code to

epoll_event evnt = {0};
evnt.data.fd = _eventfd;
evnt.events = EPOLLOUT;
if (epoll_ctl(_epollfd, EPOLL_CTL_MOD, _eventfd, &evnt) == -1)
    throw ServerError("epoll_ctl(mod)");

Now it works but It shouldn't be this way.

I don't believe this should be any difficult. What have I done wrong?

Thanks

What is `_events` initialised to? – caf Mar 19 '11 at 01:51 — caf, Mar 19 '11 at 01:51
_events was set as EPOLLIN | EPOLLET – user666412 Oct 14 '11 at 15:45 — user666412, Oct 14 '11 at 15:45

score 3 · Accepted Answer · edited Sep 05 '12 at 20:16

Works for me. For reference, here is the complete C code: it prints "eventfd_write", "1" and "DING: 1". Tested on Linux 2.6.35-30-generic #56-Ubuntu SMP.

#include <stdio.h>
#include <errno.h>
#include <sys/epoll.h>
#include <sys/eventfd.h>
#include <pthread.h>
#include <stdlib.h>

int _epollfd, _eventfd;

int init()
{
    _epollfd = epoll_create1(0);
    if (_epollfd == -1) abort();
    _eventfd = eventfd(0, EFD_NONBLOCK);
    struct epoll_event evnt = {0};
    evnt.data.fd = _eventfd;
    evnt.events = EPOLLIN | EPOLLET;
    if (epoll_ctl(_epollfd, EPOLL_CTL_ADD, _eventfd, &evnt) == -1)
        abort();
}

void *subprocess(void *arg)
{
    static const int EVENTS = 20;
    struct epoll_event evnts[EVENTS];
    while (1) {
        int count = epoll_wait(_epollfd, evnts, EVENTS, -1);
        printf("%d\n", count);
        if (count == -1)
        {
            if (errno != EINTR)
            {
                perror("epoll_wait");
                return NULL;
            }
        }

        int i;
        for (i = 0; i < count; ++i)
        {
            struct epoll_event *e = evnts + i;
            if (e->data.fd == _eventfd)
            {
                eventfd_t val;
                eventfd_read(_eventfd, &val);
                printf("DING: %lld\n", (long long)val);
                return NULL;
            }
        }
    }
}

int main()
{
    pthread_t th;
    init();
    if (pthread_create(&th, NULL, subprocess, NULL) != 0)
        abort();
    sleep(2);
    printf("eventfd_write\n");
    eventfd_write(_eventfd, 1);
    sleep(2);
}

Works for me too (3.1.10-1.16-desktop #1 SMP PREEMPT). My distro was OpenSuSE 11.3 and I'm not sure which kernel I was running ... — user666412, Sep 05 '12 at 20:13

score 1 · Answer 2 · edited Sep 24 '12 at 20:16

1

If you're using multiple threads you have to chain your call to eventfd_write at the end of each thread. This is only one option.

edited Sep 24 '12 at 20:16

Omar

16,329
10
48
66

answered Sep 03 '12 at 22:53

Talr

11
2

It's only one thread. That chaining would require a barrier before going back to `epoll_wait()`, so that one thread wouldn't wake up twice, right? – user666412 Sep 05 '12 at 20:17

linux - Can't get eventfd to work with epoll together

2 Answers2