select() max sockets

Question

Just more asynchronous stuff!

Alright, so I now have a working asynchronous socket program for my main chatting application, and it's working really well! However I have one concern..

While using select() what is the maximum number of file descriptors that I can use in each set? I've read about a limit of 1024...

If that limit is indeed hard coded and I can't FD_SETSIZE the limit any higher, should I spawn another thread once I reach that limit? Or something else? Is this even a concern?

Alright, sounds like I'm stuck with *nix if I'm going to use poll correct? — ultifinitus, Mar 18 '11 at 21:57

score 11 · Accepted Answer · answered Mar 18 '11 at 20:35

11

Yes, the FD_SETSIZE has a limit of 1024. You can easily check that by looking at the select.h header. People have tried to increase the limit, but the reports vary from "working" to "crashing" after a while. If you need that many connections, use poll instead.

A very good article to read.

answered Mar 18 '11 at 20:35

Milan

15,389
20
57
65

Thank you for the article! And the answer! – ultifinitus Mar 18 '11 at 21:08

score 3 · Answer 2 · answered Mar 18 '11 at 20:36

3

If you are programming under a Posix compliant system, you should be able to use the poll() function instead of select() and this will do away with the limit that you mention. Alternatively, you can call select() multiple times in succession but be certain to use a relatively short timeout.

answered Mar 18 '11 at 20:36

Jon Trauntvein

4,453
6
39
69

using select() multiple times wont work, since the limit lies with the FDs that are larger than the FD_SETSIZE. That is, as soon as a fd larger than 1024 is used, it will break. Read carefully the notes section on man pages. – Milan Mar 18 '11 at 20:58
It will work if you reconstruct the set for a new set of handles or if you use a different set altogether. I have already done this under windows. – Jon Trauntvein Mar 19 '11 at 13:27
For my test system, that's actually what I ended up doing. I'm thinking about simply spawning new threads for each multiple of 1024 – ultifinitus Mar 21 '11 at 04:42
@JonTrauntvein IIRC, on Windows, FD_SETSIZE is the number of sockets that can be in a set, but in Linux, it's **the maximum socket number** (plus one) that can be in a set. So if FD_SETSIZE is 1024, and you have 1025 sockets, one of them will be numbered 1025, and you can't use it. (Approximately) – user253751 Oct 02 '16 at 20:47

score 3 · Answer 3 · answered Mar 18 '11 at 20:46

3

For really large numbers of sockets look into using a library like libevent.

The library can abstract several OS-specific advanced features like /dev/poll, kqueue, epoll, and event ports. With these you can handle really vast numbers of connections.

answered Mar 18 '11 at 20:46

Zan Lynx

53,022
10
79
131

Just took a look at it, I'll give it a try! – ultifinitus Mar 18 '11 at 23:24

score 1 · Answer 4 · answered Mar 18 '11 at 20:48

1

You don't say what OS you're using, but for most, if you want to use file descriptors above 1024 with select, you can #define FD_SETSIZE to be a larger number BEFORE #including sys/socket.h. Unfortunately, this doesn't work on Linux.

answered Mar 18 '11 at 20:48

Chris Dodd

119,907
13
134
226

I had read something about that, and the server may be on several operating systems, unfortunately. – ultifinitus Mar 18 '11 at 21:13

select() max sockets

4 Answers4

Linked