What is the expected overhead of a multi-process system compared to multi-threaded application?

Question

For some operating systems (OpenBSD & ...) threads are userland-only and mostly exist for compatibility. Disregarding that type of implementation, how does the use of kernel-level threads compare to a multi-process design? What are the security implications of each? How is the performance? Complexity of development?

Answer 1

Threads share everything by default. Processes share nothing by default. So, the only difference is in the amount of sharing. Realizing that, Plan 9 and Linux provide a single system call (rfork() for Plan 9, clone() for Linux) that can create a new process, a new threads, or something in between.

Some platforms don't support fork() (e.g: Win32, Java); this makes creating a new process really expensive, which has lead to the widespread belief that threads help with performance.

Threads are lighter to create than processes, but the POSIX API allows very efficient creation of processes. Still, some systems are slow at creating new processes.

On a single CPU computer, context-switching between threads of the same process is potentially faster than between different processes (e.g: there is no need to flush the TLB). You would need to be context-switching a lot for this to have a significant impact on performance . OTOH, on a SMP machine, sharing data between threads running in different cores may mean that some data needs to move between caches (cacheline bouncing). Predicting the performance of multithreaded programs can be hard.

Security-wise, since threads share everything, it means there isn't any protection between threads, so in a sense it's like going back to the days before virtual memory OSes.

As for complexity, correctly writing multithreaded programs is harder than correctly writing non-multithreaded programs (some people have said an order of magnitude harder). What makes things worse, is that writing multithreaded programs apparently looks simpler to beginners than writing event-driven programs.