escape characters and strings

Question

I have a function that goes as:

def set_localrepo(self):

    stream = open("file1.yml", "r")
    results = yaml.load(stream)
    run_cmd = results["parameters"]["tag"]["properties"]
    config_list = ( 'sleep 200', '[sh, -xc, \"echo test\'  test\' >> /etc/hosts\"]')
    for i in range(len(config_list)):
        run_cmd.append(config_list[i])
    stream.close()
    with open("f2.yml", "w") as yaml_file:
        yaml_file.write(yaml.dump(results, default_flow_style=False, allow_unicode=True))
    yaml_file.close()

In this file, I have a file1.yml skeleton and from there I am processing and writing content from list to f2.yml.

Intended output should look like:

        properties:
        - sleep 200 
        - [sh, -xc, "echo $repo_server'  repo-server' >> /etc/hosts"]

But instead it looks like this:

        properties:
        - sleep 200 
        - '[sh, -xc, "echo $repo_server''  repo-server'' >> /etc/hosts"]'

I have tried multiple combinations with double \, single \, etc., but it dint work as I wanted it to be.

Please advise what can be done to resolve this problem. I suspect it has something to do with YAML dump utility and escape character combination.

Thanks in anticipation!

score 3 · Answer 1 · answered Jan 11 '16 at 10:48

According to your intended output, the second item is not a string but a list. So in order to be properly serialized in Python this must be a list (or tuple) as well. This means your config_list should look as follows:

( 'sleep 200', ['sh', '-xc', '"echo test\'  test\' >> /etc/hosts"'] )

After this change, the output will be:

parameters:
  tags:
    properties:
    - sleep 200
    - - sh
      - -xc
      - '"echo test''  test'' >> /etc/hosts"'

Because you disabled the default_flow_style you have that weird nested list which is equivalent to:

parameters:
  tags:
    properties:
    - sleep 200
    - [sh, -xc, '"echo test''  test'' >> /etc/hosts"']

And if you are worried about your single quotes are right as double single quotes inside single quoted strings in YAML means one single quote only.

Extra. Don't use:

for i in range(len(config_list)):
  item = config_list[i]
  # ...

Instead, use the simpler iteration pattern:

for item in config_list:
  # ...

But how do you explain this: `( 'sleep 200', '#Comment')` getting printed as `- sleep 200 - '#Comment'` — AB2328, Jan 11 '16 at 12:49
From the link I reference in the answer: Strings containing any of the following characters must be quoted. Although you can use double quotes, for these characters it is more convenient to use single quotes, which avoids having to escape any backslash \: :, {, }, [, ], ,, &, *, #, ?, |, -, <, >, =, !, %, @, \` — Salva, Jan 11 '16 at 15:49

score 1 · Accepted Answer · answered Jan 11 '16 at 10:52

I'm not a YAML expert, but I think this is expected behavior.

A replication of your program, with re-loading of the YAML file using yaml.load reveals that it has reconstructed the string correctly, as expected:

import yaml

config_list = ( 'sleep 200', '[sh, -xc, "echo $test\'  test\' >> /etc/hosts"]')
results = {'properties': []}
run_cmd = results['properties']
for i in range(len(config_list)):
    run_cmd.append(config_list[i])

with open("f2.yml", "w") as yaml_file:
    yaml_file.write(yaml.dump(results, default_flow_style=False, allow_unicode=True))

yaml_file.close()

yaml_file2 = open('f2.yml', 'r')
data = yaml_file2.read()
print(yaml.load(data))

This yields an output of

{'properties': ['sleep 200', '[sh, -xc, "echo $test\'  test\' >> /etc/hosts"]']}

Which is exactly what you would expect. Using single quotes outside and '' inside must be YAML's way of escaping the single-quote within a list item.

escape characters and strings

2 Answers2