Best way to use hydra during deep learning model inference

Question

I am currently experimenting with hydra-conf for my deep learning project. For training, i follow the way shown in the documentation:

• folder of configuration files, a main config.yml file with additional configurations groups in extra folders
• my main method is decorated with @hydra.main(...) to initialize/load the configuration

When my model is trained, i save the corresponding state dict.

Now the question is, what is the intended way of doing model inference using hydra? My idea is to use the model checkpoint as well as the run-specific config.yml file in the .hydra folder to get the constructor-parameters i need for instantiating the model, before loading the state dict.

Is there a better way to achieve my goal?

Edit 1: When trying to use the @hydra.main() approach i do not know how i would pass a dynamic config-file path to the decorator, since this would be supplied by the user (Scenario: I want to do inference on a certain model by providing the path to the config file in the training-run folder created by hydra)

Answer 1

There are many way to do it, and none of them is labeled as "best" :). What you do depends on your needs and on your constraints. You need to initialize the config object and your model and pass them, to your inference logic.

This can be done inside the same Hydra app you used to train the model initially, or in a new simpler app. You can use https://omegaconf.readthedocs.io/en/2.0_branch/ directly to load your config as you are loading your model.

If you want to recompose your config, you should use the original app configs and the overrides list (also in .hydra main), along with the https://hydra.cc/docs/experimental/compose_api. Re-composing is more complicated compares to just using the exact config.yaml saved by your training run, but offer greater flexibility.

Edit1: Not very clean: You can try to use a combination of --config-dir and --config-name (https://hydra.cc/docs/advanced/hydra-command-line-flags).

If your primary config is also named 'config` I think you will have to rename it as it will be loaded instead of the config in the generated output directory (in .hydra).

Better but require code changes: A practical alternative is something like:

@hydra.main()
def infer(cfg):
  cfg = OmegaConf.load(cfg.config_path)
  ...

$ python infer.py +config_path=outputs/2021-03-30/17-26-51/.hydra/config.yaml
...