As far as I can tell, there is no way to parameterize character strings in an AllenNLP config file --- only ints or floats

Question

So the issue is that, for using autotuning (like optuna) with AllenNLP, the suggested practice is to use, in jsonnet scripts, references to environment variables, and then to set up a study to modify those parameters.

That works fine, when the values are integers or floating points. For integers, you use std.parseInt(std.extVar(varname)), for floating point numbers, you use std.parseJson(std.extVar(varname)).

But if I want to change, say the optimization technique in my tests between "adam", "sparseadam", "adamax", adamw", etc. or change the type of RNN I am using, there does not appear to be an easy way to do that.

It would seem that you should be able to do std.extVar(varname) in that case without wrapping it inside a parseJson() or parseInt(), but that returns an error. Has anybody else had that problem and how did you get around it?

Just to add to this, I am trying this with three different string parameters. Here is the jsonnet for the first one, "bert_vocab":

local bert_vocab=std.extvar('bert_vocab');

Error message:

    486         ext_vars = {**_environment_variables(), **ext_vars}
    487 
--> 488         file_dict = json.loads(evaluate_file(params_file, ext_vars=ext_vars))
    489 
    490         if isinstance(params_overrides, dict):

RuntimeError: RUNTIME ERROR: field does not exist: extvar
    /bigdisk/lax/cox/jupyter/bert_config.jsonnet:28:18-28   thunk <bert_vocab>
    /bigdisk/lax/cox/jupyter/bert_config.jsonnet:61:22-32   object <anonymous>
    /bigdisk/lax/cox/jupyter/bert_config.jsonnet:(59:16)-(63:12)    object <anonymous>
    /bigdisk/lax/cox/jupyter/bert_config.jsonnet:(58:21)-(64:10)    object <anonymous>
    /bigdisk/lax/cox/jupyter/bert_config.jsonnet:(56:19)-(65:8) object <anonymous>
    During manifestation    

I also tried various "string escaping functions" like here (but none of the string escaping functions work either:

local bert_vocab=std.escapeStringBash(std.extvar("bert_vocab"));

I can do the following to verify that the os environment variable is set:

os.environ['bert_vocab'] returns 'bert-base-uncased'

Answer 1

(I don't know AllenNLP, only Jsonnet.)

ExtVars can be arbitrary Jsonnet values (numbers, floats, strings, arrays, objects, functions or nulls).

Judging from the examples you brought up, AllenNLP passes the parameters as strings and you need to do the parsing. In such case it should be possible to just use "naked" std.ExtVar to get a string.