How to get wandb to pass arguments by position?

Question

I am trying to explore the results of different parameter settings on my python script "train.py". For that, I use a wandb sweep. Each wandb agent executes the file "train.py" and passes some parameters to it. As per the wandb documentation (https://docs.wandb.ai/guides/sweeps/configuration#command), in case of e.g. two parameters "param1" and "param2" each agents starts the file with the command

/usr/bin/env python train.py --param1=value1 --param2=value2

However, "train.py" expects

/usr/bin/env python train.py value1 value2

and parses the parameter values by position. I did not write train.py and would like to not change it if possible. How can I get wandb to pass the values without "--param1=" in front?

score 1 · Accepted Answer · edited Feb 22 '22 at 16:56

Don't think you can get positional arguments from W&B Sweeps. However, there's a little work around you can try that won't require you touching the train.py file.

You can create an invoker file, let's call it invoke.py. Now, you can use it get rid of the keyword argument names. Something like this might work:

import sys
import subprocess

if len(sys.argv[0]) <= 1:
  print(f"{sys.argv[0]} program_name param0=<param0> param1=<param1> ...")
  sys.exit(0)

program = sys.argv[1]
params = sys.argv[2:]

posparam = []
for param in params:
  _, val = param.split("=")
  posparam.append(val)

command = [sys.executable, program, *posparam]
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = process.communicate()
sys.stdout.write(out.decode())
sys.stdout.flush()
sys.stderr.write(err.decode())
sys.stderr.flush()
sys.exit(process.returncode)

This allows you to invoke your train.py file as follows:

$ python3 invoke.py /path/to/train.py param0=0.001 param1=20 ...

Now to perform W&B sweeps you can create a command: section (reference) in your sweeps.yaml file while sweeping over the parameters param0 and param1. For example:

program: invoke.py
...
parameters:
  param0:
    distribution: uniform
    min: 0
    max: 1
  param1:
    distribution: categorical
    values: [10, 20, 30]
command:
 - ${env}
 - ${program}
 - /path/to/train.py
 - ${args_no_hyphens}

Thank you, that is definetively a possibility. I will accept the answer in a few hours if there is no more direct suggestion — R.Brä, Feb 22 '22 at 22:13
I don't think there is a direct way to pass positional arguments. Currently, we (I work for W&B) only support 4 different ways to pass arguments, `${args}`, `${args_no_hyphens}`, `${args_json}` and `${args_json_file}` as you might see in the reference doc [here](https://docs.wandb.ai/guides/sweeps/configuration#command). — Adrish Dey, Feb 22 '22 at 23:57
Thank you again! Since it was expected in this other code, which I was instructed to run with wandb, I assumed that there would be some workaround, but I since found out that there are some other problems anyway, so maybe not ;) — R.Brä, Feb 23 '22 at 09:17

How to get wandb to pass arguments by position?

1 Answers1