12

While not always true for everyone, there seems to be a majority (anecdotal from others I've spoken with) that experience faster Jest runs with --maxWorkers=50% (or some similar setting) than without it set or setting it to 100%.

Example blog on the subject

My personal experience with an 8 core machine is that if I don't set --maxWorkers then I will have 7 concurrent workers running (as expected coreCount - 1) and it will run slower than if I set --maxWorkers=50% which creates 4 concurrent workers.

Why this is happening doesn't make sense to me. i.e. allocating more resources slows this down instead of speeding it up. Can anyone explain this?

Guy
  • 65,082
  • 97
  • 254
  • 325

3 Answers3

11

As you may read in the documentation --maxWorkers in single run mode (not watch mode) by default is set to the cores available on your machine, or your CI runner minus one for the main thread.

It may be useful to adjust this in resource-limited environments like CIs but the defaults should be adequate for most use cases.

For environments with variable CPUs available, you can use percentage-based configuration: --maxWorkers=50%

In this article in Adobe Tech blog you would see a benchmarking and their explanation of why setting --maxWorkers=50% saved them some time.

In short, they explained how Jest is greedy in terms of resources, it uses all your CPU cores to run tests in parallel, which may cause peaks in memory usage and timeouts as it has to maintain a worker pool of child processes that run tests at a cost.

This article presents a benchmark that shows that may make us assume that this issue might be caused by Node.js

And that's why this might be reduced if you use half of your machine's cores, or even better sometimes by using --runInBand (or --maxWorkers=1) which is recommended in Jest's troubleshooting guide:

While Jest is most of the time extremely fast on modern multi-core computers with fast SSDs, it may be slow on certain setups as our users have discovered.

Based on the findings, one way to mitigate this issue and improve the speed by up to 50% is to run tests sequentially.

So my answer would be:

That cost of spawning and scheduling multiple jest-workers might be sometimes greater than the gains we may get with parallelization. By using multiple workers you are instantiating more objects that will load different files from the disk. That's why in small and atomic tests you don't see any (or just a small) performance gain by using --maxWorkers=50%|1.

Read more:

Fcmam5
  • 4,888
  • 1
  • 16
  • 33
  • 1
    thanks for the answer. Nobody disputes the facts that you've presented in this answer which are the same facts that I presented in the question. However, I still don't see an answer to my original question which is Why is this happening. You linked to an article and followed that with "...their explanation of why..." but that article does not answer "why." – Guy Feb 21 '23 at 18:24
  • 1
    I updated my answer with my conclusion: The cost of spawning multiple instances might be sometimes greater than the gain from parallelization – Fcmam5 Feb 21 '23 at 19:01
  • Fcmam5, that still does explain the Why. i.e. how does that factor into using 4 out of 8 cores instead of 7 out of 8? How does the cost of spawning relate to 4 that doesn't apply to 7? – Guy Feb 23 '23 at 03:36
  • 1
    It applies, but if u have an overall execution time `T1`, it'd be splitted to `n` workers, which is exactly what we would all expect, that `TW = T1/n`, and that `TW < T1` so the more cores (`n`) takes the load the smaller `TW` (the time that each worker takes) is, then your execution time would be the MAX of all of your `TW`. In reality, there' r some tasks [under the hood](https://github.com/facebook/jest/tree/main/packages/jest-worker) to spawn each worker, And the greater is `n` the greater is the time it'd take to spawn each worker – Fcmam5 Feb 23 '23 at 09:15
  • This seems to be a performance issue win Jest, and there are many open issues in their repository, but even if they fix it somehow we would still have some time "wasted" waiting for mutexes over other resources as having more workers means more concurrency and more time take to spawn and retrieve results form each one of the workers – Fcmam5 Feb 23 '23 at 09:32
  • Great answer. The reason why it happens is presumably because of overhead from other processes necessary to running your OS or other software. It's the same reason that you don't run a virtual machine using the same number of cores as your physical machine has - it doesn't make it faster, but instead slower. – Slbox Feb 24 '23 at 01:08
0

I have never used Jest or a testing suit similar to it. But reading your question, it seems you are trying to find an optimized way to run your test suit and understand why there is a discrepancy in the usage with 50% CPU vs 90%.

Looking at the article, it seems unclear if this machine that is utilized is a shared machine with other resources (Development environment) or a dedicated server where you are running the tests.

I find more questions what hardware are you running (are you mimicking the article)? You mention an 8-core machine, but is it x86, ARM? Is hyperthreading enabled in your bios? Are you running services like antivirus/malware programs on the device (these run by default in the background)? What OS are you on, Windows Ubuntu, for these tests? What's the amount of memory in usage (Are you using DDR4 vs DDR5)? What disks are you using NVMe?

The article seems to be trying to benchmark between different devices, which is fine, and you will note the results are still varied as those devices run on different architectures (not to mention manufacturers).

Usually, there is no optimal answer to why --maxWorkers=50% works better than, let's say, 70%, 80%, or 90%. If you run these on a server with 24 dedicated cores, 256GB Memory, and NVME Write incentive disks. I can say that your same test suit would run optimally at closer to 80% than at 50% (provided your tests are not big objects). It is also likely that Jest has some bad configuration in CPU usage (which is a possibility on how they read files, store, and run tests in parallel, where after a certain amount of threads, the complexity of parallel computing does not work), or just the cost of spawning child threads is high as mentioned by @Fcmam5.

I don't think anyone can answer your question about why 4 CPU threads are better than 7 CPU threads without re-running these tests in a similar environment. The answer could be simply that the DISK buffer is limited or that the Memory you hold for the object tests is being limited. Maybe some background processes are running that are sharing those last three threads, which leads to longer wait times than Jest running just on the idle four threads.

This is more relevant to the system in question, and you can only optimize it for your System. You can look for further optimization by tweaking the system in question. If you are simply trying to collect data I would recommend running these tests on dedicated servers for more conclusive data.

Mohammed
  • 121
  • 2
  • 8
-1

If you check your CPU usage and memory on the processes you'll see when you run jest it creates x new threads (or workers) with usually the same amount of memory being used, that's because it loads pratically everything between those threads, in this case this can be harmful on CI envs and also could mean it would consume tons of memory without a real need. If you specify a smaller amount of workers it will span less threads(obviously) and still will use the same amount of memory that it was using before, so it will have much lower processing and memory consumption.

Thiago Przyczynski
  • 1
  • 1