14

I have been using YAML as configuration file format in several applications, and all went well except one thing: when my program needs to write/modify a config variable in a YAML config file, it destroys formatting and comments by loading and dumping the entire file/structure.

(Well, there is another problem with YAML actually. Most users, many of them are not programmers, will be tripped over the details of YAML rules, like the significance of whitespace in some places. But this is not a major gripe.)

What I would prefer is a YAML loader/dumper which can do round-trip parsing (preserving all whitespaces & comments), or some other human-readable serialization format which has such parser. I'm even considering using Perl document and PPI, since PPI is a round-trip safe parser. Or perhaps PPI can be bent to deal with YAML or similar formats? I'd rather not use XML, I'd resort to INI+(JSON|YAML|... for key values) before that.

Any advice or pointers?

Steven Haryanto
  • 741
  • 5
  • 13

3 Answers3

13

If you are using block structured YAML and Python is acceptable, you can use the Python package¹ ruamel.yaml which is a derivative of PyYAML and supports round trip preservation of comments:

import sys
import ruamel.yaml

inp = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

yaml = ruamel.yaml.YAML()

code = yaml.load(inp)
code['name']['given'] = 'Bob'

yaml.dump(code, sys.stdout)

with result: 

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Note that the end-of-line comments are still aligned.

Instead of normal list and dict objects the code consists of wrapped versions² on which the comments attached.

¹ Install with pip install ruamel.yaml. Works on Python 2.6/2.7/3.3+. Disclaimer: I am the author of that package.
² ordereddict is used in case of a mapping, to preserve ordering

Anthon
  • 69,918
  • 32
  • 186
  • 246
3

Yeah, you and everyone who thought wow, yaml sounds cool, simply put, it doesn't exist, yet

update: you probably want to use Config::General, its apache config format (xmlish)

No, PPI is not general purpose tool, if you want BNF-ness, you want to use Marpa

Of all INI/JSON/YAML/XML, XML probably has the best editor support for non-programmers (sounds crazy)

glebm
  • 20,282
  • 8
  • 51
  • 67
wantpretty
  • 146
  • 3
2

One approach to this is using "lenses". See Augeas for one implementation.

Hynek -Pichi- Vychodil
  • 26,174
  • 5
  • 52
  • 73
  • 1
    As a note: Augeas has no support for YAML (yet). – raphink Dec 12 '11 at 21:21
  • @Raphlink: It tells us something about YAML or Augeas. I personally think it tells more about YAML than Augeas ;-) – Hynek -Pichi- Vychodil Dec 13 '11 at 11:34
  • I like YAML very much for complex configuration files or as an inter-process mean of sharing serialized data, but it's a complex format. The JSON lens is already complex, but YAML adds even more complex stuff: several ways to write arrays, references and their associated pointers, etc. – raphink Dec 13 '11 at 21:54