How to use multiline command in 'script:' with YAML?

Question

I have a repository that uses Travis CI, and in the .travis.yml there I have this line:

script:
- vim -Nu <(cat <<-EOF
  set nocompatible |
  filetype off
  EOF
  ) -c 'Script' > /dev/null

Sadly this doesn't work, as this is transformed into a single line and is executed like this:

vim -Nu <(cat <<-EOF set no compatible | filetype off | EOF ) -c 'Script' > /dev/null

This makes the EOF tag not working, as EOF needs to be in a single line. An alternative would be to just use normal quotes like this:

script:
- vim -Nu <(cat 'set nocompatible |
  filetype off
  ) -c 'Script' > /dev/null

Which works, and is fine, but I feel there must be a way to insert newlines into a .travis.yml. I have an alternative now, but I may not in the future. So how do you do it?

Anthon · Accepted Answer · 2017-08-11T07:03:30.083

In YAML you can specify newlines in a scalar by using "" quoting and escaping the newlines (\n), or, and that is more natural for your case, by using a literal style block scalar:

script:
- |
  vim -Nu <(cat <<-EOF
  set nocompatible |
  filetype off
  EOF
  ) -c 'Script' > /dev/null

This is a scalar starting with a line with a | (pipe symbol), followed by multiple lines for which the line-breaks are preserved.

The lines are normally indented (exception: a single top-level literal style block scalar).
After the | there can be modifiers: 1-9, used when your first line starts with spaces; +, - to influence stripping of final newlines (normally collapsed into one).

There is no `if` statement in my example, so I am not sure what you are doing. Maybe you should just post a question instead of remark, that, in this form, is of little use to anyone. — Anthon, Jun 14 '21 at 06:24

serghei · Answer 2 · 2023-07-21T23:14:35.913

Your question relates to the YAML language standard, which is used in the configuration files of many CI/CD systems, including Travis CI, GitLab CI, GitHub Actions, and others. However, it's worth noting that not all CI/CD systems support the YAML standard fully, and they might also add their own extensions.

YAML has special block styles for handling line breaks. In your case, the folded block style, denoted by the “>” symbol might be useful. It allows long lines to be broken up for readability while retaining the semantics of the original long line. Here's an example:

script:
    - >
      valgrind
      --read-var-info=yes
      --error-exitcode=1
      --fullpath-after=
      --track-origins=yes
      --leak-check=full
      --num-callers=20
      --suppressions=$(pwd)/tests/zephir_parser.3.7.0.sup
      $(phpenv which php)
          -d variables_order=EGPCS
          run-tests.php
              -p $(which php)
              -d extension=$(pwd)/modules/zephir_parser.so
              -d variables_order=EGPCS
              -g "FAIL,XFAIL,BORK,WARN,LEAK,SKIP"
              --offline
              --show-diff
              --set-timeout 120

In this style, each line break is replaced with a space. Indentation in each line will be ignored. A line break will be inserted at the end.

Note: The backslash “\” must not be used here, as it will be treated as a line continuation. Thus, the following example is wrong:

script:
    - >
      valgrind \               # this line wrong
      --read-var-info=yes

Otherwise it will be treated as valgrind \ --read-var-info=yes.

However, it's important to note that different CI/CD systems may have their peculiarities in processing configuration files. For instance, GitHub Actions uses a line folding style (“>-”), which allows long lines to be placed on multiple lines, but processes them as a single line. Here's an example:

  - name: Configure (x64 Debug)
    run: >-
      cmake
      -S .
      -Bbuild
      -DCMAKE_BUILD_TYPE=Debug
      -DCPPCHECK=ON
      -DWARNINGS_AS_ERRORS=ON
      -DCMAKE_INSTALL_PREFIX=/opt/my-program
      -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

In this case, after processing, all lines are concatenated into one.

Note: With line folding lines are left aligned, using the same indentation at the beginning of the line.

Can you throw a pipe in the middle of this? Or would that be bad? — Chaim Eliyah, Apr 06 '18 at 00:22
@ChaimEliyah YAML doesn't interpret the contents of the scalar beyond the newline after the first pipe, everything else is take "as is" (until you dedent). — Anthon, May 25 '18 at 10:39
I use ``\`` as usual, and it breaks. Why isn't ``\`` required in this case? — Minh Nghĩa, Aug 31 '20 at 19:43

score 9 · Answer 3 · answered Aug 05 '21 at 15:58

9

What you are looking for is |. Here is a very clear example to refer to.

   build:
     commands:
       - echo "${CODEBUILD_BUILD_ARN}"
       - |
         if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProjectDev-" >/dev/null; then
           yarn run build-dev;
         fi
       - |
         if expr "${CODEBUILD_BUILD_ARN}" : ".*build/MyProject-" >/dev/null; then
           yarn run build-prod;
         fi

answered Aug 05 '21 at 15:58

Shivam Anand

952
1
10
21

This seems to repeat other answers, but thanks for showing how to use this syntax in a larger context, it's quite helpful. – joanis Aug 05 '21 at 17:15

How to use multiline command in 'script:' with YAML?

3 Answers3

Linked