Why doesn't PHP pcntl_fork run more than two child processes in parallel?

Question

I understand how fork() works at a high level and the pcntl_fork() wrapper, but for some reason in my environment PHP only runs two child processes at a time. Take for example this simple code:

<?php
for ($i = 1; $i <= 5; ++$i) {
    $pid = pcntl_fork();
    if ($pid === -1) {
        print "Could not fork!\n";
        exit(1);
    }
    if (!$pid) {
        print "-In child $i\n";
        sleep(1);
        print "In child $i\n";
        exit($i);
    } else {
        print "Parent: forked $i\n";
    }
}

while (pcntl_waitpid(0, $status) != -1) {
    $status = pcntl_wexitstatus($status);
    echo "Child $status completed\n";
}

The output I expect is something like this with total time around 1 second:

Parent: forked 1
-In child 1
Parent: forked 2
-In child 2
Parent: forked 3
-In child 3
Parent: forked 4
-In child 4
Parent: forked 5
-In child 5
In child 1
In child 2
In child 3
In child 4
In child 5
Child 1 completed
Child 2 completed
Child 3 completed
Child 4 completed
Child 5 completed

But what I actually get is this, with total execution time around 3.5 seconds:

Parent: forked 1
Parent: forked 2
Parent: forked 3
Parent: forked 4
Parent: forked 5
-In child 2
-In child 1
In child 2
Child 2 completed
-In child 3
In child 1
Child 1 completed
-In child 4
In child 3
Child 3 completed
-In child 5
In child 4
Child 4 completed
In child 5
Child 5 completed

So it appears that only two child processes are actually running at any given time. I can't find any explanation for this behavior...

When running the test on a production system which is Docker on a native Linux host I get the expected result, but why can't I reproduce it with the exact same container on my WSL2 host?

System Information

I'm running this test in a very recent version of Docker Desktop via WSL2 on Windows 11.
My system has 12 CPU cores (20 with hyperthreading - nproc prints "20" from both the WSL2 hostand from inside the docker container).
I have not changed the defaults in Docker Desktop or WSL config or used any resource control flags on the Docker container so CPU should not be restricted in any way.

Running Ubuntu, I get a list of Parent: forked n, then -In child n, then In Child n and finally a set of Child n completed — Nigel Ren, Jul 12 '22 at 06:57
I'm wondering what your usecases are for using PCNTL. Do you have precautions in place for long running or stuck subprocesses? You are describing you are using docker, why not let docker manage the processes for you? Especially if you have any intentions to move this to a docker orchestrator like kubernetes. — Leroy, Jul 12 '22 at 07:04
I've implemented a setup with a root process, a dispatcher and 1-N workers using PHP. Your interpretation is faulty that PHP doesn't run multiple child processes. Please try running `htop` in the same container you're running these processes, I guess at some place there are CPU restrictions in place. — Ulrich Eckhardt, Jul 12 '22 at 07:46
@NigelRen I'm running Ubuntu as well, only via WSL.. thanks for the sanity check though. — ColinM, Jul 12 '22 at 19:11
@Leroy sure there are different ways to do it but this is a legitimate use of pcntl. The number of workers needed depends on factors that aren't easily known by the container orchestrator. — ColinM, Jul 12 '22 at 19:12
@UlrichEckhardt `htop` shows 20 CPU bars, consistent with `nproc` which shows 20 on both the host and when run inside the container using `docker exec`. — ColinM, Jul 12 '22 at 19:14
Okay, that doesn't explain it then. Some further thoughts: What if you run the code on Windows? Also, but I'm not sure here, isn't there a Docker implementation for Windows you could run this code in? Just for completeness, what does your Dockerfile look like? — Ulrich Eckhardt, Jul 12 '22 at 19:40
@UlrichEckhardt Docker Desktop is the official implementation for Windows and it integrates tightly with WSL so I'm running the latest and greatest Docker implementation available. By starting with a clean Dockerfile I narrowed it down to XDebug.. — ColinM, Jul 12 '22 at 20:42

ColinM · Answer 1 · 2022-07-12T21:15:38.253

So after running the code with the expected result on a simplified Docker image with only PHP and pcntl installed, I discovered the behavior is related to XDebug.

I had PhpStorm open and listening for debug connections (with no breakpoints set) so I believe what was happening was each child process was connecting to the debugger and the debugger can only handle two connections at a time. Either that or it has something to do with the connection established by the parent process being closed in a child process.

So if you are using pcntl and XDebug together, expect weird results.. Turning off the debugger in my IDE or disabling XDebug via PHP CLI yields the expected results.

php -dxdebug.remote_enable=0 \
    -dxdebug.default_enable=0 \
    -dxdebug.remote_autostart=0 \
    script.php

score 2 · Answer 2 · answered Jul 12 '22 at 20:54

Running this on a native Linux box (i3-6100)....

Parent: forked 1
-In child 1
Parent: forked 2
-In child 2
Parent: forked 3
-In child 3
Parent: forked 4
-In child 4
Parent: forked 5
-In child 5
In child 1
In child 2
In child 3
In child 4
In child 5
Child 3 completed
Child 4 completed
Child 5 completed
Child 1 completed
Child 2 completed

I would have been surprised if it behaved exactly as you predicted. When a process forks, it creates 2 runnable processes....as to which gets scheduled first....with the CFS they are likely to stay associated with the same core....but it will still be effectively random. And PHP does have a lot of work to do (including stuff which will result in yielding the CPU) when it starts - even though the fork should mean that most of this can be shortcutted. Then there is the sequence in which buffers get flushed to the tty.

Having said all that, your results are a very long way from your expectation. This appears to be a result of the MS-Windows scheduler (which IME is very lumpy).

Why doesn't PHP pcntl_fork run more than two child processes in parallel?

System Information

2 Answers2