How do I read a custom serialized YAML object (written by Rails) with Python?

Question

I am working with a Rails database that contains serialized values in one column. These values should have been regular Hashes, but due to improperly sanitizing the parameters they were stored as HashWithIndifferentAccess or Parameters. For example, one column entry looks like this:

--- !ruby/object:ActionController::Parameters
parameters: !ruby/hash:ActiveSupport::HashWithIndifferentAccess
  windowHeight: 946
  documentHeight: 3679
  scrollTop: 500
permitted: false

I want to read this with Python's yaml implementation, but when I try to do so, I get:

*** yaml.constructor.ConstructorError: could not determine a constructor for the tag '!ruby/object:ActionController::Parameters'
  in "<unicode string>", line 1, column 5:
    --- !ruby/object:ActionController::P ...
        ^

So, for some reason it expects a constructor. But quite obviously, the value itself is just a regular dictionary. How can I still read it?

score 1 · Accepted Answer · answered Jul 16 '18 at 18:05

1

You can use the add_constructor(loader, node) function of the PyYAML parser, which lets you implement custom constructors for object types that it does not recognize.

In that constructor, the function loader.construct_pairs(node) can be called to obtain key-value tuples from the original node contents. Using a dictionary comprehension, we can create the original dictionary.

Since the entry is nested, we have to apply the constructor to both object types.

A complete example would be the following:

import yaml

def convert_entry(loader, node):
    return { e[0]: e[1] for e in loader.construct_pairs(node) }

yaml.add_constructor('!ruby/hash:ActiveSupport::HashWithIndifferentAccess', convert_entry)
yaml.add_constructor('!ruby/object:ActionController::Parameters', convert_entry)

yaml.load(input_string)

This is somehow documented but it's hard to find a lot of examples.

answered Jul 16 '18 at 18:05

slhck

36,575
28
148
201

Any chance of getting the Rails code fixed to properly YAMLizied `params`? They probably want to say `params.to_unsafe_h.to_yaml` instead of `params.to_yaml`. – mu is too short Jul 16 '18 at 18:53
1

@muistooshort Yes, that's exactly what we've done now for new entries—slowly migrating the old ones to the correct format, but it takes a while. – slhck Jul 16 '18 at 19:33
1

I've come across this problem before (https://stackoverflow.com/a/50545931/479863), might make sense to switch to JSON or something along the way but that's probably out of scope. – mu is too short Jul 16 '18 at 19:42

How do I read a custom serialized YAML object (written by Rails) with Python?

1 Answers1