Running more than 4 jobs in parallel with CircleCI

Question

If I have a .circleci/config.yml file like so:

version: 2
jobs:
  build-node8:
    docker:
      - image: oresoftware/lmx-circleci:8
    steps:
      - checkout
      - run: ./scripts/circleci/run.sh
  build-node9:
    docker:
      - image: oresoftware/lmx-circleci:9
    steps:
      - checkout
      - run: ./scripts/circleci/run.sh
  build-node10:
    docker:
      - image: oresoftware/lmx-circleci:10
    steps:
      - checkout
      - run: ./scripts/circleci/run.sh
  build-node11:
    docker:
      - image: oresoftware/lmx-circleci:11
    steps:
      - checkout
      - run: ./scripts/circleci/run.sh
  build-node12:
    docker:
      - image: oresoftware/lmx-circleci:12
    steps:
      - checkout
      - run: ./scripts/circleci/run.sh

there are 5 jobs listed here, but when the builds start, only 4 jobs run in parallel. Is there a way to run more than 4 jobs in parallel is there a hard limit there?

My guess is that under workflows, I can change the parallelism level?

workflows:
  version: 2
  build_nodejs:
    parallelism: 5
    jobs:
      - build-node8
      - build-node9
      - build-node10
      - build-node11
      - build-node12

perhaps this requires a paid account tho?

if you look at the docs https://circleci.com/docs/2.0/configuration-reference/ there is no 'parallelism" option for workflows, only for each job, and I am not sure what that means — , May 30 '19 at 21:10

score 3 · Accepted Answer · answered Jun 07 '19 at 19:58

Short Answer:

CircleCi lets you run as many jobs in parallel as you want, as long as your payment plan has enough containers to serve each job.

I suspect that your plan only has 4 containers. You can check to see how many containers you have in the settings tab in CircleCi

In the below example I have a total of 2 containers available: 1 paid + 1 free. So right now at max I can only run 2 jobs in parallel. I can pay an extra $50 per month per container to add additional containers thoughs.

Additional Details:

This article gives a great overview of how to configure circle ci jobs to run in parallel (and it actually has an example where 5 jobs run in parallel). https://circleci.com/blog/decrease-your-build-times-by-running-jobs-in-parallel-with-workflows/

Regarding the config file code snippet you pasted in your question - It looks fine (Tho you don't need the parallelism: 5 flag, since circle will use all available plan capacity automatically)

Can you please check to see how many containers are in your plan and then report back?

FYI - CircleCi container and concurrent job plan info: https://circleci.com/pricing/

Running more than 4 jobs in parallel with CircleCI

1 Answers1