Do system calls execute in the context of a software interrupt handler in entirety?
I mean, some system calls like read() could take a long time to return, against the policy that ISR should be very short in execution time. Are system calls offloaded to other threads? How does that work?
[A reference to any kernel is fine]
Please read this quoted text of how a system call normally works:
The system call handler gains control when a user program starts a system call. The system call handler changes the protection domain from the caller protection domain, user, to the system call protection domain, kernel, and switches to a protected stack.
The system call handler then calls the function supporting the system call. The loader maintains a table of the currently defined system calls for this purpose.
The system call runs within the calling process, but with more privilege than the calling process. This is because the protection domain has changed from user to kernel.
The system call function returns to the system call handler when it has performed its operation. The system call handler then restores the state of the process and returns to the user program.
Now, there are two types of kernels regarding context switching during a system call. Preemptive and non-preemptive kernels(roughly speaking; as this sometimes also applies to threads/processes executing a critical section of code).
The former is common among RTOS(Real-time OSs) while the later is common among the most common OSs we know.
In preemptive kernels the scheduler is allowed to opt-out a thread executing a system call for the favor of a thread with a higher priority.
The latter doesn't allow this kind of preemption, as in; if a thread is currently executing a system call all others should wait till it finishes its kernel-mode work.
To sum it up, normally in a non-RT system, if a thread executes a system call, it should finish its work before being dispatched.
Now notice something, some system calls may be blocking!
What does this mean? It means they'll halt till an Interrupt happens, when then they'll be back in execution till they're done doing their work.
An example of that is read(), where it'll block till the data a user requests is ready; meanwhile other threads can be scheduled to run but once an interrupt comes to that read() it'll start running again and no one can dispatch it(again, only in non-preemptive environments) till it returns to user land the results.
The syscalls run on most kernels inside an ISR. Take a quick glance at a former release of Linux and you will notice the int $Ox80 to invoke the kernel. This solution which is probably the simplest from a kernel development point of view, has a strong drawback: as long as running the ISR; interrupts are disabled. Disabling interrupts too long sucks because it's obvious your system won't be reactive (it delays external events, it doesn't reschedule on time, ...).
Preemption, as Adel explained in his answer is a smart solution. But whenever the kernel choose to preempt a thread because of an unavailable ressource, it has generally already spent a lot of time with interrupts disabled.
Are system calls offloaded to other threads?
You're right. Interrupt-threads and/or threaded kernel is an even smarter solution. Kernels like Solaris and Mac OS X prefers to have very simple ISRs which just wakeup high priority interrupt threads. Therefore the ISRs are reduced to the minimum processings, and the time the system runs with interrupts disabled is strongly decreased. Because these interrupt-threads have an high priority, they are likely to run at the return of the ISR. What is nice is interrupts will be enabled again, and therefore an even higher priority work wouldn't be delayed. With a threaded kernel, such as Linux in its recent releases, multiple things can be done inside the kernel, and despite one blocks, other process are still able to enter the kernel.
Hope this help!
First, some terminology as used in Linux. In Linux, the kernel can be running in several contexts:
/proc file, ...System calls start executing with interrupts enabled (either because they are reenabled explicitly (e.g: when using x86's sysenter), or because they were never disabled at all (e.g: when using an x86's trap gate, which Linux' int 0x80 does)). So, the kernel can be interrupted by an interrupt while it is executing a system call.
Some system calls may sleep (block), waiting for something. In that case, the kernel can switch to another task (process/thread), and the asleep task will be woken up later.
In non-preemptive kernels, a task cannot preempt another task that is running (i.e. actually doing work, not sleeping) in process context. In preemptive kernels, higher priority tasks can preempt lower priority tasks that are running in a non-atomic process context. This prevents high context-switch latencies, e.g: an audio program may need to be scheduled very frequently, a preemptive kernel prevents other tasks' heavy system call processing from messing with this (which could otherwise cause audio artifacts, clicks, ...)
