Quoting and escaping for mix cmd aliases

Question

I can, in principle, run any shell commands as mix tasks by defining aliases in mix.exs that start with cmd. But I am struggling with white spaces, quoting, and backslash escaping. Here is some non-working examples:

defp aliases do
  f1: ["cmd echo \"two  spaces  between\""],
  f2: ["cmd echo 'two  spaces  between'"],
  f3: ["cmd echo two\ \ spaces\ \ between"],
  f4: ["cmd echo two\\ \\ spaces\\ \\ between"],
end

They all just produce string "two spaces between", not what I expected or aimed for, "two spaces between":

$ for t in f1 f2 f3 f4; do mix $t; done
two spaces between
two spaces between
two spaces between
two spaces between

So how can I pass quoted strings to shell commands? And more generally, how is the alias cmd target processed before it is run by shell or is it executed directly? What's the logic of removing single and double quotes and backslashes from the command string?

Update: More failing cases:

  q1: ["cmd echo \"one  two  three\""],         # "one two three"
  q3: ["cmd echo \\\"one  two  three\\\""],     # "one two three"
  q5: ["cmd echo \\\\\"one  two  three\\\\\""], # "one" (!!!)

  f5: [
    """
    cmd echo 'two  spaces  between'
    """
      ],

Obviously, I would prefer being able to pass both kinds of quotes for the intended effect but there seems to be some undocumented, and in this context overaggressive and unnecessary sanitation of input string in operation.

I am running this with Elixir 1.11.1.

Answer 1

Don't know if this is applicable to your situation, but when you define the tasks in local functions it seems to work better:

  defp aliases do
    [
      f1: &f1/1,
      f2: &f2/1
    ]
  end

  defp f1(_) do
    Mix.shell().cmd("echo 'two  spaces  between'")
  end

  # or if you just want to display some text
  defp f2(_) do
    Mix.shell().info("two  spaces  between")
  end

Output:

$for t in f1 f2; do mix $t; done
two  spaces  between
two  spaces  between

Answer 2

Mystery Solved

What is going on? It is about how the task arguments are parsed. And the solution is to quote the whole shell command after cmd to pass it to the shell without unintended transmogrifications.

The listed aliases f1, f2, f3, f4 with non-intended effects

defp aliases do                                # BAD!!
  f1: ["cmd echo \"two  spaces  between\""],
  f2: ["cmd echo 'two  spaces  between'"],
  f3: ["cmd echo two\ \ spaces\ \ between"],
  f4: ["cmd echo two\\ \\ spaces\\ \\ between"]
end

are equivalent to running from shell

$ mix cmd echo "two  spaces  between"           # BAD!!!
$ mix cmd echo 'two  spaces  between'
$ mix cmd echo two  spaces  between
$ mix cmd echo two\ \ spaces\ \ between

They all output two spaces between.

Running

$ mix cmd "echo \"two  spaces  between\""       # GOOD!!!
$ mix cmd "echo 'two  spaces  between'"
$ mix cmd "echo two \\ spaces \\ between"

gives the intended output one two three.

So, the answer is that the shell commands should be quoted in their entirety like this:

defp aliases do                                 # GOOD!!!
  ok1: ["cmd \"echo \\\"two  spaces  between\\\"\""],
  ok2: ["cmd \"echo 'two  spaces  between'\""],
  ok3: ["cmd \"echo two\\ \\ spaces\\ \\ between\""],
end

or equivalently (using the ~S sigil of Aleksei to simplify the quoting):

defp aliases() do                               # GOOD!!!
  ok1: [~S|cmd "echo \"two  spaces  between\""|],
  ok2: [~S|cmd "echo 'two  spaces  between'"|],
  ok3: [~S|cmd "echo two\ \ spaces\ \ between"|],
end

Then everything works like fancied:

$ for t in ok1 ok2 ok3; do mix $t; done
two  spaces  between
two  spaces  between
two  spaces  between

Peeking Behind The Curtain

Looking at elixir source code, file lib/mix/lib/mix/tasks/cmd.ex module Mix.Tasks.Cmd function run/1 you can see how the command is passed to shell. It basically comes to this:

Mix.shell().cmd(Enum.join(args, " ")

where args are the shell command arguments after mix cmd.

So command mix cmd echo 'two spaces' or alias target ["cmd echo 'two spaces'] results to calling Mix.Tasks.Cmd.run(["echo", "two spaces"]) and running

Mix.shell().cmd("echo two  spaces")

where of course shell does not care how many spaces you have between the words. The working mix cmd "echo 'two spaces'" or alias target ["cmd \"echo 'two spaces'\""] results to calling Mix.Tasks.Cmd.run(["echo 'two spaces']) and is put together as

Mix.shell().cmd("echo 'two  spaces'")

giving the intended effect.

More Useful Example

The solution to the actual problem I was struggling to accomplish:

defp aliases() do
  "check-formatted": [~S/cmd "bash -c 'mix format --check-formatted $(git ls-tree -r --name-only HEAD | egrep \"[.]exs?$\")'"/],
  # ...
end

(on my machine this requires running through bash to produce a list of file names for mix format instead of one huge string of file names).

This gets ugly really quick, so for more complicated scenarios @zwippie's solution of using Elixir functions to define aliases and therein using Mix.shell().cmd() to run the steps that need to be accomplished by running external programs is really the superior way to go.

Answer 3

Quotes are eaten by the shell itself, and I was always only good at guessing here :)

Apparently, this would work:

foo: [~S|cmd echo \\\"two\\ \\ spaces\\ \\ between\\\"|]

mix foo
#⇒ "two  spaces  between"