7

I want to execute a shell script while handling stdout and stderr output. Currently I execute commands using Process.run, with shell=false and three pipes for stdin, stdout and stderr. I spawn fibers to read from stdout and stderr and log (or otherwise process) the output. This works pretty well for individual commands, but fails horribly for scripts.

I could simply set shell=true when calling Process.run, but looking at the Crystal source it seems that merely prepends "sh" to the commandline. I've tried prepending "bash" and it didn't help.

Things like redirection (>file) and pipes (e.g. curl something | bash) don't seem to work with Process.run

For example, to download a shell script and execute it, I tried:

cmd = %{bash -c "curl http://dist.crystal-lang.org/apt/setup.sh" | bash}

Process.run(cmd, ...)

The initial bash was added in the hope that it would enable the pipe operator. It doesn't seem to help. I also tried executing each command separately:

script.split("\n").reject(/^#/, "").each { Process.run(...) }

But of course, that still fails when a command uses redirection or pipes. For example, the command echo "deb http://dist.crystal-lang.org/apt crystal main" >/etc/apt/sources.list.d/crystal.list simply outputs:

"deb http://dist.crystal-lang.org/apt crystal main" >/etc/apt/sources.list.d/crystal.list`

It might work if I used the `` backticks method of execution instead; but then I wouldn't be able to capture the output in real time.

Sod Almighty
  • 1,768
  • 1
  • 16
  • 29
  • you need to move the pipeline into the `bash -c` in the first case to get it to invoke. You're fighting against `shell=false`, though, which is the thing that would allow you to use shell syntax directly. – Anya Shenanigans Feb 18 '16 at 14:57
  • @Petesh Sadly, setting `shell=true` doesn't help any. – Sod Almighty Feb 18 '16 at 15:16
  • @Petesh Moving the pipeline into the `bash -c` call seems to work, however. So...why doesn't `shell=true` work? Surely it's pretty much the same thing? – Sod Almighty Feb 18 '16 at 15:24

3 Answers3

13

The problem is a UNIX problem. The parent process must be capable to access the STDOUT of the child process. Using a pipe you must start a shell process that will run the whole command, including the | bash and not just curl $URL. In Crystal this is:

command = "curl http://dist.crystal-lang.org/apt/setup.sh | bash"
io = MemoryIO.new
Process.run(command, shell: true, output: io)
output = io.to_s

Or if you want to duplicate what Crystal does for you:

Process.run("sh", {"-c", command}, output: io)
Julien Portalier
  • 2,959
  • 1
  • 20
  • 22
5

I'm basing my understanding on reading the source code of the run.cr file. The behaviour is very similar to other languages in how it deals with commands and arguments.

Without shell=true, the default behaviour of Process.run is to use the command as the executable to run. This means that the string needs to be a program name, without any arguments, e.g. uname would be a valid name as there's a program on my system called uname in /usr/bin.

If you ever got behaviour of successfully using %{bash -c "echo hello world"} with shell=false, then something is wrong - the default behaviour should have been to try to run a program called bash -c "echo hello world", which is unlikely to exist on any system.

Once you pass in 'shell=true', then it does sh -c <command>, which will allow strings like echo hello world as a command to work; this will also allow redirections and pipelines to work.

The shell=true behaviour can generally be interpreted as doing the following:

cmd = "sh"
args = [] of String
args << "-c" << "curl http://dist.crystal-lang.org/apt/setup.sh | bash"
Process.run(cmd, args, …)

Note that I'm using an array of arguments here - without the array of arguments, you don't have any control over how the arguments are passed into the shell.

The reason why the first version, with or without shell=true doesn't work is because the pipeline is outside the -c, which is the command you're sending to bash.

Anya Shenanigans
  • 91,618
  • 3
  • 107
  • 122
  • I think the problem was due to the fact I was splitting the command string into `cmd` (a string) and `args` (an array) and then calling `Process.new(cmd, args, ...)`. Even with `shell=true`, it didn't work - probably because I had already split the `|` into an argument of its own... – Sod Almighty Feb 18 '16 at 22:39
  • Basically, I ended up calling `Process.new("curl", ["http://dist.crystal-lang.org/apt/setup.sh", "|", "bash"], shell: true)`. And of course this didn't work. To support `shell=true`, I needed to modify my wrapper function to avoid splitting the command into words if `shell` is true. – Sod Almighty Feb 18 '16 at 22:41
2

or if you want to call a shell script and get the output i just tried with crystal 0.23.1 and it's work ! 

def screen
    output = IO::Memory.new
     Process.run("bash", args: {"lib/bash_scripts/installation.sh"}, output: output)
     output.close
    output.to_s
end
Oliver
  • 21
  • 2