How to use callable targets with dataclass arguments in Hydra?

Question

Is it possible to define a target using Structured Configs to avoid redefining all the parameters?

def good(config: Config):
    pass

def bad(param1, param2):
    pass

@dataclass
class Config:
    param1
    param2
    _target_: Any = good
    # _target_: Any = bad
    # _target_: str = 'Config.also_good'

    def also_good(self):
        pass

What type annotation should I use for _target_ in case of a class, function, or method? When I used Any I got

omegaconf.errors.UnsupportedValueType: Value 'function' is not a supported primitive type
    full_key: _target_

Hi Emre, normally the `_target_` has type `str`, e.g. `_target_: str = "__main__.good"` or `_target_: str = "my_module.bad"`. The value of `_target_` should be the dotpath used to look up the callable that you are targeting. See the `PostGreSQLConfig` example on Hydra's [Structured Configs example](https://hydra.cc/docs/advanced/instantiate_objects/structured_config/) webpage. — Jasha, Jan 27 '22 at 01:13
I am not sure what you mean by asking "Is it possible to define a target using Structured Configs to avoid redefining all the parameters?" Could you please clarify what you mean by "avoid redefining the parameters?" If you are talking about avoiding duplication of "param1" in different places (e.g. in the function signature and in the dataclass body), see https://github.com/facebookresearch/hydra/issues/1936. — Jasha, Jan 27 '22 at 01:24
I mean using a target with one dataclass parameter, rather than a bunch of primitive parameters for each field of the dataclass. I saw that issue but did not see how hydra-zen solves the problem. Feel free to post an answer showing how it does. — Emre, Jan 27 '22 at 01:32
I am still confused about what you are trying to achieve. What is the relationship between the good / bad / also_good functions in your example? Have you tried running a hydra app and, if so, are you getting an error message? More details in your question would be helpful. — Jasha, Jan 27 '22 at 01:39
The good ones accept dataclasses (`also_good` through `self`), and bad one does not. When I try to run it clearly indicates it is looking for the primitive parameters because the error messages say they are missing, and names them. My dataclasses have many fields, and it is unwieldy to keep the signatures in sync. Also, the documentation says target could be callable. — Emre, Jan 27 '22 at 01:54

score 3 · Accepted Answer · answered Jan 31 '22 at 02:34

The _target_ type should be str. Here is an example using the instantiate API with a structured config:

# example.py
from dataclasses import dataclass

from hydra.utils import instantiate


def trgt(arg1: int, arg2: float):
    print(f"trgt function: got {arg1}, {arg2}")
    return "foobar"


@dataclass
class Config:
    _target_: str = "__main__.trgt"  # dotpath describing location of callable
    arg1: int = 123
    arg2: float = 10.1


val = instantiate(Config)
print(f"Returned value was {val}.")

Running the script:

$ python example.py
trgt function: got 123, 10.1
Returned value was foobar.

Notes:

The _target_ field must be a string describing the "dotpath" used to look up a callable (__main__.trgt in the example above). Other typical dotpaths would be e.g. my_module.my_function, builtins.range numpy.random.randn, etc. The instantiate function uses this dotpath to find the callable object that will be instantiated (the trgt function, in this example).
The fields of the structured config (except for the _target_ field and other special reserved fields such as _recursive_, _convert_ and _args_) will be passed as keyword arguments to the looked-up callable. In this example, the function trgt is called with keyword arguments arg1=123 and arg2=10.1. This is equivalent to the python code trgt(arg1=123, arg2=10.1).
If you define a structured config (using @dataclass or @attr.s), you can specify a default value for each field. For example, the default value for field arg1 above is 123. Calling instantiate(config) will result in an exception if config has any fields whose values are missing. Typically, a structured config's missing values will be filled in during Hydra's config composition process.

References:

How to use callable targets with dataclass arguments in Hydra?

1 Answers1