Why is Go considered partially preemptive?

Question

I am trying to get a better understanding of the definition preemptive and cooperative in the context of Go. Wiki states for preemptive multitasking

In computing, preemption is the act of temporarily interrupting an executing task, with the intention of resuming it at a later time. This interrupt is done by an external scheduler with no assistance or cooperation from the task.

Wiki states "external scheduler". I guess it means dispatcher to be more specific since afaik the scheduler only is responsible for choosing the next process in line.

Go is often referred to as partially preemptive since sync-points/preemption-points are only at function calls and not at any given instruction. That makes sense. But as the wiki definition states, preemptive is done by an external scheduler.

But isn't every process or task preemptive since the CPU can stop executing any process mid-execution in order to switch to another process? Any clarification is appreciated!

Addendum

The only explanation I could come up with is that we talk about different preemption levels. One for processes, and one for kernel/user-threads. In that case the CPU scheduler selects the next process but the Go scheduler is responsible for the goroutines/threads.

Answer 1

My answer is built on top of Andrew and Daniel's answers, and entirely based on this talk.

Since go 1.14, the go scheduler is non-cooperative pre-emptive. Each go routine is pre-empted after a certain time slice. It's 10ms in go 1.19.1.

In the talk I mentioned earlier, starting 20:35, you can find the history about how the scheduler used purely co-operative pre-emption in 1.0, then compiler baked in pre-emption with go 1.2, and finally the current nature - non-coperative pre-emption.

Answer 2

Your addendum is correct. There are different schedulers working here. One is the OS scheduler and one on the application level. Check out this article. It's exactly what you are looking for:

As we discussed in the first post, the OS scheduler is a preemptive scheduler. [...] The kernel is making decisions and everything is non-deterministic.

And this:

The Go scheduler is part of the Go runtime, and the Go runtime is built into your application. This means the Go scheduler runs in user space, above the kernel. The current implementation of the Go scheduler is not a preemptive scheduler but a cooperating scheduler. Being a cooperating scheduler means the scheduler needs well-defined user space events that happen at safe points in the code to make scheduling decisions.

Conlusion, there are two different schedulers. One for the process, one for Goroutines.

Answer 3

There are a few things here. First the Wikipedia article is talking about operating system level preemption which may not exactly apply. Go-routine scheduling is not handled by the OS but by the Go runtime.

In essence processes/threads are handled by the OS/hardware and are preemptive. The Go runtime can run different "go routines" on the same thread. (In fact this is one thing that makes Go unique - easily allowing you to create an application with millions of concurrent go-routines.)

Originally go-routines were not preemptive but surprisingly this caused few problems but a long-running loop that did not encounter preemption points could hog a thread. But thanks to some great work by Austin Clements this was addressed a couple of years ago (Go 1.15?). So I believe go-routines are fully preemptable at least on the more popular architectures, but you may encounter old comments that state otherwise.

I hope this helps but to answer your specific questions:

But isn't every process or task preemptive...?

Yes, but they are not go-routines.

the CPU scheduler selects the next process but the Go scheduler is responsible for the goroutines/threads.

No, the OS (using CPU/MMU hardware facilities) schedules processes and threads. The Go scheduler only decides which go-routine runs next on the specific thread(s) it controls, and it never controls more than GOMAXPROCS (non-blocked) threads.