Multiple commands in the same build step in Google Cloud Builder

Question

I want to run our automated backend test suite on Google Cloud Builder environment. However, naturally, I bumped into the need to install various dependencies and prerequisites within the Cloud Builder so that our final test runner (php tests/run) can run.

Here's my current cloudbuild.yaml:

steps:

  - name: 'ubuntu'
    args: ['bash', './scripts/install-prerequisites.sh', '&&', 'composer install -n -q --prefer-dist', '&&', 'php init --overwrite=y', '&&', 'php tests/run']

At the moment, the chaining of multiple commands doesn't work. The only thing that's executed is the bash ./scripts/install-prerequisites.sh part. How do I get all of these commands get executed in order?

score 25 · Answer 1 · answered Jun 25 '19 at 22:57

25

A more readable way to run the script could be to use breakout syntax (source: mastering cloud build syntax)

steps:
- name: 'ubuntu'
  entrypoint: 'bash'
  args:
  - '-c'
  - |
    ./scripts/install-prerequisites.sh \
    && composer install -n -q --prefer-dist \
    && php init --overwrite=y \
    && php tests/run

However, this only works if your build step image has the appropriate deps installed (php, composer).

answered Jun 25 '19 at 22:57

guille

551
4
7

1

This doesn't work if you have more steps after this one? – Faraji Anderson Jul 31 '20 at 00:23
Can you provide an example of what you're trying to do? Each build step pulls the original container. If you want to re-use the ubuntu container from this example with the prereqs installed across multiple steps, I would suggest to build and push that custom image and reference that instead. – guille Aug 04 '20 at 18:24

score 16 · Accepted Answer · answered May 14 '19 at 01:25

16

You have 2 options to achieve this at the moment I believe:

create a script that has the sequence of commands you'd like and call the script directly:

# cloudbuild.yaml
steps:
  - name: 'ubuntu'
    args: ['./my-awesome-script.sh']

# my-awesome-script.sh
/usr/bin/env/bash

set -eo pipefail

./scripts/install-prerequisites.sh
composer install -n -q --prefer-dist
php init --overwrite=y
php tests/run

Call bash -c with all the commands you'd like to follow:

steps:
  - name: 'ubuntu'
    args: ['bash', '-c', './scripts/install-prerequisites.sh && composer install -n -q --prefer-dist && php init --overwrite=y && php tests/run']

answered May 14 '19 at 01:25

Gustavo Hoirisch

1,637
12
19

Question author here: For any future readers, this difficulty was exactly the reason to make me switch to Github Actions. Try it for yourself as well. Thank me later. – Dzhuneyt Nov 23 '19 at 15:42
Option #1 requires bash e.g. `args: ['bash', './scripts/install-prerequisites.sh']` works. See https://cloud.google.com/cloud-build/docs/create-custom-build-steps – ruhong Feb 17 '20 at 03:28

score -3 · Answer 3 · answered Apr 26 '19 at 08:24

See:

By default, build steps run sequentially, but you can configure them to run concurrently.

The order of the build steps in the steps field relates to the order in which the steps are executed. Steps will run serially or concurrently based on the dependencies defined in their waitFor fields.

A step is dependent on every id in its waitFor and will not launch until each dependency has completed successfully.

So you only separate command as each step.

Like this.

steps:
  - name: 'ubuntu'
    args: ['bash', './scripts/install-prerequisites.sh']
    id: 'bash ./scripts/install-prerequisites.sh'
  - name: 'ubuntu'
    args: ['composer', 'install', '-n', '-q', '--prefer-dist']
    id: 'composer install -n -q --prefer-dist'
  - name: 'ubuntu'
    args: ['php', 'init', '--overwrite=y']
    id: 'php init --overwrite=y'
  - name: 'ubuntu'
    args: ['php', 'tests/run']
    id: 'php tests/run'

By the way, Can using ubuntu image run php and composer command?

I think that you should use or build docker image which can run php and composer command.

The composer docker image is here.

steps:
- name: 'gcr.io/$PROJECT_ID/composer'
  args: ['install']

Running the commands as individual build steps is not an option, because each one of them runs in a "clean" ubuntu environment. They don't inherit one another's installed software stack. — Dzhuneyt, Apr 26 '19 at 16:16
How can they suggest this then? https://cloud.google.com/cloud-build/docs/building/build-nodejs#configuring_nodejs_builds Where build depends on install? @Dzhuneyt — vonGohren, Sep 25 '20 at 16:20
Cloud Build mounts the source folder to the docker and npm install adds files to mounted folder, not to docker image. Next docker run also has the source folder mounted and npm can find installed files in source folder. — Null, Nov 30 '21 at 02:41

Multiple commands in the same build step in Google Cloud Builder

3 Answers3

Linked