why epoll accepted new fd and spawn new thread doesn't scale well?

Question

I have seen a lot of argument about epoll accepted new fd and spawn new thread for read and write on it's own thread doesn't scale well? But how it doesn't scale well? What if every connection has heavy processing like:-

• doing database transaction
• doing heavy algorithm work
• waiting for other things to completed.

If my purpose definitely just want to do the thing inside the program(no more fancy routing to other connection to do stuff), and do not spawn new thread for read/write io. It might be hanging forever just because of one function waiting for something right? If this is the case how epoll scale well if do not spawn new thread?

epoll_wait(...);

// available to read now
recv(....);

// From here if i don't spawn thread, the program will be hanging. What should I do?
processing algorithm work.....// At least 3 secs to do the job.


continue;

Answer 1

AFAIU, epoll(7) does not spawn new threads by itself (see also pthreads(7)...). You need some other call (using pthread_create(3) or the underlying clone(2) system call used by pthread_create...) to create threads.

Read more about the C10K problem (which today should be called C100K) and some pthread tutorial. But it looks like your program could be compute-intensive, not IO-bound. So the bottleneck might be computer power (then you cannot get scalability with just multi-threading on a single computer node; you need distributed computing)

Threads are quite heavy resources. So you want to have some thread pool and have only a few dozens of active (i.e. runnable) threads. See this.

Be also aware of other multiplexing system calls (such as poll(2)), of non-blocking IO (fcntl(2) with O_NONBLOCK), of asynchronous IO (see aio(7)).

I recommend using some existing event-loop based library (look into libev, libevent, Glib, Poco, Qt, ... or for HTTP mostly: libonion on the server side, libcurl on the client side). Look also into 0mq.

The concepts related to callbacks, continuations, CPS could be useful and improve your thinking.

Languages like Go and its Goroutines could be helpful.

It might be hanging forever ....

That should not happen if you design your program carefully (of course having event loops using something like poll or epoll_wait - with a limited delay of less than a second and probably prefering non-blocking IO).

Probably, spending a few weeks learning more about Operating Systems concepts should be worthwhile. Also understanding most system calls (listed in syscalls(2)) after having read more about Linux programming (e.g. the old ALP book, or something newer) should be preferable. Perhaps you don't need something as sophisticated as epoll (because using just poll might be enough).