Using dictionary rather than parameter.yml for Kedro

Question

Is there a way to use dictionary rather than using a yaml config for parameters.yml? I want to keep it as a Python Object because my IDE can then track the dependency easily. For my parameters, I am injecting functions in it.

If i need to use yml, I will have to use

def steps1(x, func1):
   func1 = eval(func1)

And this will break the refactoring features easily.

score 0 · Answer 1 · answered Aug 21 '20 at 10:42

You could overwrite the params property in your src.package_name.run.ProjectContext so that it uses a Python dictionary instead of the config loader. You’re also welcome to write up your custom ConfigLoader and use that instead (by overriding _create_config_loader), but that’s probably more effort. Please bear in mind that parameters in Kedro are meant to be “as dumb as possible” though, as it’s considered static configuration and it’s better separated out of code. What you describe, with expressions, sounds more suited for nodes.

I don't know what's the best practice of dynamicly build a pipeline. Are you suggesting I should create nodes for all those function, and use expression to only add the nodes that need to be executed? — mediumnok, Aug 21 '20 at 17:29

Using dictionary rather than parameter.yml for Kedro

1 Answers1