5

I just saw the new extension fields in Docker Compose v3.4 and tried to use it in combination with partially override.

My compose files looks like this:

version: '3.4'

x-environment:
  &default-environment
  - FOO=foo
  - BAR=bar

services:
  myservice:
    [...]
    environment:
      << : *default-environment
      - BAZ=baz

When I try to run it, I get:

ERROR: yaml.parser.ParserError: while parsing a block mapping
expected <block end>, but found '-'

(for my line "<< : *default-environment")

Any ideas?

Munchkin
  • 4,528
  • 7
  • 45
  • 93

1 Answers1

4

You are trying to use the merge key (specified for outdated YAML version 1.1) with sequences. This cannot work since the merge key is not a new syntax, but merely a special key that can be used in mappings.

When the parser parses the content of environment, it sees this line first:

<< : *default-environment

This line contains an implicit key, <<, and a value for that key, *default-environment. Thus, the parser decides that the value of environment is a mapping.

Now on the next line, the parser encounters a -, which is an indicator for a sequence item. Since the parser is currently parsing a mapping, it cannot handle a sequence item indicator and thus stops with an error.

Note that the merge key is not defined for current YAML version 1.2 and will be explicitly deprecated for the upcoming 1.3 version. Therefore, its usage is discouraged in general. Moreover, there is no way of using it with sequences, as you discovered.

flyx
  • 35,506
  • 7
  • 89
  • 126
  • Thank you! I found a workaround for that special case: I use the merge key with `env_file:` (where I define *FOO* and *BAR*) and then, within my actual service, I use `environment:` (where I define *BAZ*). // When merge key will be deprecated, how will one do this kind of things in newer versions? – Munchkin Dec 01 '17 at 21:51
  • @Munchkin The first answer is that you do *not* do this kind of things in YAML because YAML is a serialization language, not a programming language. The second answer is that we are working on an optional *„transformation extension*“ which *may* be released as spec along with YAML 1.3 and will be able to do this (also with sequences) and some other things that are often asked about on SO. – flyx Dec 02 '17 at 08:51
  • Thanks, I understood, my question was about the **correct** "merge key" usage (with mappings): When It's deprecated soon, how will it be replaced? You say the replacement (*transformation extension*) only **may** be released in 1.3? So it may be possible, that I'm not able anymore to merge mappings? Isn't this a downgrade? – Munchkin Dec 04 '17 at 11:53
  • 1
    Well since the merge key was never part of the spec itself, it always only was a suggestion, and poorly defined (fails to specify interaction with explicit tags). It also kind-of violates YAML's concept of tags mapping to data structures (it maps to a processing instruction instead). We want to discourage its implementation and usage for these reasons. The transformations extension is not seen as a replacement, but as a proper way of doing data transformations in YAML which were never considered to be part of the language before. Currently we are prototyping to check whether it is feasible. – flyx Dec 04 '17 at 13:00
  • 1
    Our reasoning is 1. *„this was a bad idea and should have never been defined, so we declare it as deprecated to discourage implementation“*. 2. *„But it shows that there is a genuine desire for such a feature, so let's try and properly include it in the language“*. If our tests are successful, we'll release it alongside YAML 1.3. If not, we keep our position of *„this should not be part of the language“*. – flyx Dec 04 '17 at 13:37
  • Well I just saw docker-compose uses pyyaml@3.12, which was last updated 15 month ago (08/16) and this lib works with YAML@1.1 ... so if the compose-guys won't switch to another yaml parser, I think I can use the merge key for quite a long time :D – Munchkin Dec 04 '17 at 13:47