How to create two fields from the same group in hydra-python?

Question

My config files structure:

config
├── train_dataset
│   ├── adobe5k.yaml
│   ├── my_train_data1.yaml
│   └── cifar10.yaml
├── valid_dataset
│   ├── adobe5k.yaml
│   └── cifar10.yaml
└── config.yaml

And my config.yaml is:

# other configs
...

defaults:
  - train_dataset: adobe5k
  - valid_dataset: adobe5k

As you see, I have 2 fields named valid_dataset and train_dataset in my config, whose value is selected from its own config group. What should I do to make values of the two fields selected from the same group?

score 0 · Answer 1 · answered Jul 16 '21 at 08:20

0

You can use Defaults List interpolation to achieve that. The Default List page high level information and you find a more practical example here for some details.

defaults:
  - train_dataset: adobe5k
  - valid_dataset: ${train_dataset}

This way, it will be enough for you to override train_dataset and the validation dataset would match it automatically (unless you override it too).

answered Jul 16 '21 at 08:20

Omry Yadan

31,280
18
64
87

What to do if I want valid_dataset to be another value from the train_dataset group? More broadly, is it possible to create just a dataset group and assign files interchangeably to train_dataset or valid_dataset? – rs_ Jul 28 '21 at 06:50
Please ask a different question. I don't think I fully understand your question and I am 100% sure I can't answer it in this comment form. – Omry Yadan Jul 28 '21 at 07:42
Asked a new question here : https://stackoverflow.com/questions/68556877/hydra-how-to-assign-config-files-from-same-group-to-two-different-fields – rs_ Jul 28 '21 at 08:25

score 0 · Accepted Answer · answered Aug 27 '21 at 13:31

0

You can use a config group more than once. Assume that you moved your dataset configurations into a single datasets directory. Then you can re-use them as follows.

- defaults:
  - datasets@train_dataset: adobe5k
  - datasets@valid_dataset: adobe5k

answered Aug 27 '21 at 13:31

Marten

1,336
10
16

Thanks! That's exactly what I want. – why Sep 20 '21 at 06:32

How to create two fields from the same group in hydra-python?

2 Answers2