Efficiently listening on multiple sockets even when file descriptors are not available

Question

Let's say that I have two libraries (A and B), and each has one function that listen on sockets. These functions use select() and they return some event immediately if the data has arrived, otherwise they wait for some time (timeout) and then return NULL:

A_event_t* A_wait_for_event(int timeout);
B_event_t* B_wait_for_event(int timeout);

Now, I use them in my program:

int main (int argc, char *argv[]) {
// Init A
// Init B
// .. do some other initialization
    A_event_t *evA;
    B_event_t *evB;
    for(;;) {
        evA = A_wait_for_event(50);
        evB = B_wait_for_event(50);
        // do some work based on events
    }
}

Each library has its own sockets (e.g. udp socket) and it is not accessible from outside.

PROBLEM: This is not very efficient. If for example there is a lot of events waiting to be delivered by *B_wait_for_event* these would have to wait always until *A_wait_for_event* timeouts, which effectively limits the throughput of library B and my program.

Normally, one could use threads to separate processing, BUT what if processing of some event require to call function of other library and vice verse. Example:

if (evA != 0 && evA == A_EVENT_1) {
    B_do_something();
}
if (evB != 0 && evB == B_EVENT_C) {
    A_do_something();
}

So, even if I could create two threads and separate functionality from libraries, these threads would have to exchange events among them (probably through pipe). This would still limit performance, because one thread would be blocked by *X_wait_for_event()* function, and would not be possible to receive data immediately from other thread.

How to solve this?

Don't make one thread per library: make one thread per independent thing you want to do. — , Apr 04 '12 at 14:02
I am not sure that I understand that quite right. What if libraries are not thread safe? How can I call function of library A from thread T2 if T1 actually initialized that library and is waiting now for events? — Sasa, Apr 04 '12 at 14:11

score 3 · Answer 1 · answered Apr 04 '12 at 14:38

This solution may not be available depending on the libraries you're using, but the best solution is not to call functions in individual libraries that wait for events. Each library should support hooking into an external event loop. Then your application uses a single loop which contains a poll() or select() call that waits on all of the events that all of the libraries you use want to wait for.

glib's event loop is good for this because many libraries already know how to hook into it. But if you don't use something as elaborate as glib, the normal approach is this:

Loop forever:
- Start with an infinite timer and an empty set of file descriptors
- For each library you use:
  - Call a setup function in the library which is allowed to add file descriptors to your set and/or shorten (but not lengthen) the timeout.
- Run poll()
- For each library you use:
  - Call a dispatch function in the library that responds to any events that might have occurred when the poll() returned.

Yes, it's still possible for an earlier library to starve a later library, but it works in practice.

If the libraries you use don't support this kind of setup & dispatch interface, add it as a feature and contribute the code upstream!

Your solution is also very good. A looked in the API deeper and I found a function that returns socket, which is used to signal internal events in library. Now I am able to integrate this library in my event loop. Thank you very much! — Sasa, Apr 06 '12 at 16:11

score 2 · Accepted Answer · 2012-04-04T15:42:40.017

(I'm moving this to an answer since it's getting too long for a comment)

If you are in a situation where you're not allowed to call A_do_something in one thread while another thread is executing A_wait_for_event (and similarly for B), then I'm pretty sure you can't do anything efficient, and have to settle between various evils.

The most obvious improvement is to immediately take action upon getting an event, rather than trying to read from both: i.e. order your loop

Wait for an A event
Maybe do something in B
Wait for a B event
Maybe do something in A

Other mitigations you could do are

Try to predict whether an A event or a B event is more likely to come next, and wait on that first. (e.g. if they come in streaks, then after getting and handling an A event, you should go back to waiting for another A event)
Fiddle with the timeout values to strike a balance between spin loops and too much blocking. (maybe even adjust dynamically)

EDIT: You might check the APIs for your library; they might already offer a way to deal with the problem. For example, they might allow you to register callbacks for events, and get notifications of events through the callback, rather than polling wait_for_event.

Another thing is if you can create new file descriptors for the library to listen on. e.g. If you create a new pipe and hand one end to library A, then if thread #1 is waiting for an A event, thread #2 can write to the pipe to make an event happen, thus forcing #1 out of wait_for_event. With the ability to kick threads out of the wait_for_event functions at will, all sorts of new options become available.

score 1 · Answer 3 · edited May 23 '17 at 12:27

1

A possible solution is to use two threads to wait_for_events plus boost::condition_variable in "main" thread which "does something". An alike but not exact solution is here

edited May 23 '17 at 12:27

Community

1
1

answered Apr 04 '12 at 14:21

megabyte1024

8,482
4
30
44

That is a nice solution, except that even if one slave thread completes, the main thread has to pass event to second slave threads, which is still blocked by wait_for_events. – Sasa Apr 04 '12 at 14:26

Efficiently listening on multiple sockets even when file descriptors are not available

3 Answers3