Using multiple models with repeated keywords in `hydra`

Question

I'm pretty new to hydra and I'm trying to better understand the config.yaml file. I'm undertaking a deep learning experiment where I have two separate models, an embedding network and a simple fully connected neural network. The first one is going to create features, and the second is basically fine-tuning the results. I would like to quickly access some parameters relative to the configuration for both models. For now I just tried to incorporate everything in the same config.yaml file

parameters_embnet:
    _target_: model.EmbNet_Lightning
    model_name: 'EmbNet'
    num_atom_feats: 200
    dim_target: 128
    loss: 'log_ratio'
    lr: 1e-3
    wd: 5e-6
    
data_embnet:
    _target_: data.CompositionDataModule
    dataset_name: 's'
    batch_size: 64
    data_path: './s.csv'
    
wandb_embnet:
    _target_:  pytorch_lightning.loggers.WandbLogger
    name: embnet_logger
    
    
trainer_embnet:
    max_epochs: 1000
    
    
parameters_nn:
_target_: neuralnet.SimpleNeuralNetwork_Lightning
input_size: 200
lr: 1e-3
wd: 5e-6
loss: 'log_ratio'

data_nn:
    _target_: neuralnet.nn_dataset_lightning
    batch_size: 128

wandb_nn:
    _target_:  pytorch_lightning.loggers.WandbLogger
    name: neuralnet_logger
    
trainer_nn:
    max_epochs: 150

but trying to use such configuration results in a ConstructorError since some keys (like lr) are duplicated across the two models. Now, I'm just wondering whether this is the correct way to proceed, or if I should set up multiple config.yaml files and what's the most optimal way to do that.

score 1 · Answer 1 · answered Nov 05 '22 at 17:32

1

It's not clear exacty what you are trying to do, but it is not legal to have the same key mutliple times.

This block in particular looks like it both have the same keys multiple times and is incorrectly indented.

parameters_nn:
_target_: neuralnet.SimpleNeuralNetwork_Lightning
input_size: 200
lr: 1e-3
wd: 5e-6
loss: 'log_ratio'
lr: 1e-3

answered Nov 05 '22 at 17:32

Omry Yadan

31,280
18
64
87

I just corrected that, so that was the reason why I was getting the `ConstructorError`? Cause I still have two separate `lr` but they belong to different models now. – James Arten Nov 05 '22 at 19:17
YAML is a nesting of maps and lists. The maps cannot have the same keys multiple times (at the same level). – Omry Yadan Nov 06 '22 at 02:58

score 0 · Answer 2 · answered Dec 10 '22 at 06:55

Based upon OP comment: I would like to quickly access some parameters relative to the configuration for both models I infer a question related to a concept of common params, a base param set plus customization over a couple of key concepts.

See my post in Use a parameter multiple times in hydra config file. I give a secondary example that may answer your implied question.

.. Otto

Using multiple models with repeated keywords in `hydra`

2 Answers2