Editing existent YAML and duplicating entries through PyYAML

Question

I have a YAML with the following structure:

- description: 'DUMMY_JOB'
  sequence:
    commands:
      - description: Metadata
        script: ASDF
      - description: Login
        jobref:
        args: ASDF
        group: ''
        name: ASDF
        nodeStep: 'true'
      - description: Deployment
        jobref:
        args: ASDF
        group: ''
        name: ASDF
        nodeStep: 'true'
      - description: Logout
        jobref:
        group: ''
        name: ASDF
        nodeStep: 'true'
    keepgoing: false
    strategy: node-first
  uuid: 'UNIQUE_ID'

This YAML is used as a job description inside Rundeck, what I have to do is basically duplicate the commands edit the data (the olds commands) and the new ones to reflect new parameters.

I'm loading it through PyYaml with:

jobStream   = open("example.yaml", 'r')
jobMap      = yaml.safe_load(jobStream)

And accessing it with:

jobMap[0]['sequence']['commands']

Brings back the data I need. However if I simply duplicate it through a merge like:

commandsA = jobMap[0]['sequence']['commands']
commandsB = jobMap[0]['sequence']['commands']
jobMap[0]['sequence']['commands'] = commandsA + commandsB

If I try do yaml.dump(jobMap) it won't have the commands duplicated but the IDs show up:

- *id001
- *id002
- *id003
- *id004

This YAML is invalid, even when just removing `(...)`, as `keepgoing: false` cannot have the same indent level as the sequence element above it. Delete the "YAML" in your post, copy in the YAML you have, select all the lines that constitute the YAML and press `{}`, if there are no tab characters in the YAML, the formatting will be fine instead of what you have now. — Anthon, Jul 11 '17 at 15:20
Any particular reason you are using PyYAML? Is your source following the outdated YAML 1.1 standard? — Anthon, Jul 11 '17 at 16:16
@Anthon, just fixed the YAML format, thanks a lot! And I'm using PyYaml only because it was the way I found how to do it, I'm open to some other approach if recommended. — Christian Bernini, Jul 11 '17 at 16:59
If the files have comments that you want to preserve, or if you care about the key ordering, etc then you should look at my `ruamel.yaml` implementation. — Anthon, Jul 11 '17 at 19:35

score 0 · Accepted Answer · answered Jul 11 '17 at 15:11

0

The elements from the list commandsA and from commandsB are dictionaries, if you copy those you'll get multiple reference in your structure to the same items, leading to anchors (&) and aliases (*)

The quickest way to get the YAML that you want is probably using deepcopy:

from copy import deepcopy

jobMap[0]['sequence']['commands'] = deepcopy(commandsA) + deepcopy(commandsB)

answered Jul 11 '17 at 15:11

Anthon

69,918
32
186
246

Thanks a lot, I was able to get it working through deepcopy! – Christian Bernini Jul 11 '17 at 17:11
It is possible to switch off automatic generation of anchors and aliases as well, but for manipulation of files like you do, this is shorter and conceptually much simpler. – Anthon Jul 11 '17 at 19:36

Editing existent YAML and duplicating entries through PyYAML

1 Answers1