SO_REUSEPORT and multiple listeners in the same thread

Question

Background references...

https://lwn.net/Articles/542629/

https://blog.n0p.me/2018/02/2018-02-20-portsharding/

I'm curious how well this feature performs with multiple listeners on the same thread?

For example, suppose my server enforces a limit of 100 connections. Using a framework like ASIO, can it simply create a pool of 100 connection objects, with each one listening asynchronously until it accepts a connection? It's really more like having 100 single-connection servers.

Will multiple listeners on the same thread have the same or worse performance than a single listener?

[edit] I think the listen backlog may be an issue. Even if the backlog is set to one, a second connection can be queued right after the first is accepted, and closing the port will probably RST the connection instead of migrating it to another listen backlog.

Answer 1

Ok, I wrote up a test program to see what happens and the acceptor backlog queue is definitely a problem.

With 100 acceptors and 100 connectors, roughly 60% of connectors would get distributed to a unique acceptor, while the remainder ended up in the backlogs of those same acceptors (depth set to 1). If the acceptor immediately closed the port after its initial accept, then the percentage climbed randomly with the remainder being rejected.

Without kernel support for single-shot accept, this approach can't work. It's a shame since it would simplify some implementation models, where connection objects can handle their own accepts like they can already do with connects.