2

I have an application that runs multiple Python scripts in order. I can run them in docker-compose as follow:

command: >
  bash -c "python -m module_a &&
  python -m module_b &&
  python -m module_c"

Now I'm, scheduling the job in Nomad, and added the below command under configuration for Docker driver:

command = "/bin/bash"
args = ["-c", "python -m module_a", "&&","
      "python -m module_b", "&&",
      "python -m module_c"]

But Nomad seems to escape &&, and just runs the first module, and issue exit code 0. Is there any way to run the multiline command similar to docker-compose?

Daniel Mann
  • 57,011
  • 13
  • 100
  • 120
seemvision
  • 242
  • 1
  • 2
  • 12

1 Answers1

2

The following is guaranteed to work with the exec driver:

command = "/bin/bash"
args = [
  "-c",                                                  ## next argument is a shell script
  "for module; do python -m \"$module\" || exit; done",  ## this is that script.
  "_",                                                   ## passed as $0 to the script
  "module_a", "module_b", "module_c"                     ## passed as $1, $2, and $3
]

Note that only a single argument is passed as a script -- the one immediately following -c. Subsequent arguments are arguments to that script, not additional scripts or script fragments.

Even simpler, you could run:

command = "/bin/bash"
args = ["-c", "python -m module_a && python -m module_b && python -m module_c" ]
Charles Duffy
  • 280,126
  • 43
  • 390
  • 441
  • Hey, thanks, I actually tried this, but either methods results in exit code 2: `Exit Code: 2, Exit Message: "Docker container exited with non-zero exit code: 2"` – seemvision Oct 20 '17 at 12:45
  • I don't suppose making it `-xc` rather than `-c` (thus logging commands run to stderr) might be enlightening? – Charles Duffy Oct 20 '17 at 12:51
  • (Without a log saying otherwise, "exit code: 2" could mean that one of the Python modules was successfully started and then exited with that status due to an environmental or installation difference between your prior environment and this one.) – Charles Duffy Oct 20 '17 at 12:56
  • Not sure if that's the issue. I can run both with `docker run -it /bin/bash -c 'python -m module_a && python -m module_b && python -m module_c'` and it seems to run both. Maybe Nomad doesn't provide full shell? – seemvision Oct 20 '17 at 13:48
  • With this (array) usage, Nomad doesn't provide any shell at all. The `/bin/bash` target on your executable is used. – Charles Duffy Oct 20 '17 at 13:49
  • Any other recommendation on how to run these modules in order? Even if I can run them as separate periodic tasks. – seemvision Oct 20 '17 at 13:51
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/157141/discussion-between-charles-duffy-and-seemvision). – Charles Duffy Oct 20 '17 at 13:52