perl two consecutive back ticked shell command do not run

Question

I am trying to write perl script for managing amazon ec2 instances. In part of my code, I have two shell commands, and when I invoke them, the first runs but not the second. I cannot seem to find a good explanation for this. Here is the code:

$run_instances = 'ec2-run-instances ami-8e1fece7 -k mykey -t t1.micro';
$outp = `$run_instances`;
$outp =~ /INSTANCE\s+(i-\w+)\s/;
$instance_id = $1;
$describe_instances = "ec2-describe-instances $instance_id";
$outp = `$describe_instances`;
$outp =~ /(ec2-(\d+-\d+-\d+-\d+).\S+)/;

The problem is $outp here has the output of $run_instances. For some time I could not understand why I was getting the wrong output; then I realised that $describe_instances command does not run.

I looked at the value of $describe_instances, called that from the Linux shell, and it worked fine. I called it from another Perl script, and it worked fine.

Then I gave $outp the output that is captured when $run_instances runs ($outp = "INSTANCE ......"). It worked, so I got the idea that somehow when these two commands run consecutively the second does not run.

One more thing to note is that when I put above code in a loop every time $run_instances works but $describe_instances does not.

I would be really happy if you could provide some light on this :)

Thank you

Answer 1

Your program has a few red flags. I discuss them below and at the end suggest a better way to write your code.

Red Flag 1

The code from your question attempts to execute external commands with backticks and assumes success. You should always check the status of any call to the operating system. A failed `$command`—also known as qx// or readpipe—makes itself known in one or more ways depending on whether the command executed and failed or whether attempted execution of the command failed:

• The value of the special variable $? is non-zero.
• In cases of failed execution
  • the special variable $! contains a description of the failure.
  • the operator returns the undefined value in scalar context or the empty list in list context.

With Perl, `$command` executes a subshell in order to execute $command, so the subshell may be the program that executes and fails, e.g., for bad command syntax.

Exactly one command in the example below succeeds on my machine.

#! /usr/bin/env perl

use strict;
use warnings;

my @commands = (
  "bad syntax (",
  "does-not-exist",
  "/etc/passwd",
  "perl --no-such-option",
  "perl -le 'print q(Hello world)'",
);

foreach my $c (@commands) {
  print "Capturing output of command $c...\n";
  my $output = `$c`;

  if ($? == 0) {
    print "  - command executed successfully!\n";
  }
  elsif ($? == -1) {
    print "  - command failed to execute: \$!=$!\n";
  }
  else {
    print "  - command exited with status " . ($? >> 8) . "\n";
  }

  print "  - value of \$output is ",
        (defined $output ? "" : "un"), "defined\n\n";
}

Output:

Capturing output of command bad syntax (...
sh: Syntax error: "(" unexpected
  - command exited with status 2
  - value of $output is defined

Capturing output of command does-not-exist...
Can't exec "does-not-exist": No such file or directory at ./runcmds line 16.
  - command failed to execute: $!=No such file or directory
  - value of $output is undefined

Capturing output of command /etc/passwd...
Can't exec "/etc/passwd": Permission denied at ./runcmds line 16.
  - command failed to execute: $!=Permission denied
  - value of $output is undefined

Capturing output of command perl --no-such-option...
Unrecognized switch: --no-such-option  (-h will show valid options).
  - command exited with status 29
  - value of $output is defined

Capturing output of command perl -le 'print q(Hello world)'...
  - command executed successfully!
  - value of $output is defined

Red Flag 2

Note for example the line of output

Can't exec "does-not-exist": No such file or directory at ./runcmds line 16.

I didn't write the code that generated this warning. Instead, it happened automatically because I enabled the warnings pragma with the line use warnings. Had you enabled warnings, Perl would have already given you at least a hint about the cause of your problem, so I assume you didn't enable warnings.

This is another red flag. Warnings are there to help you. Always enable them for any non-trivial program.

Red Flag 3

Finally, you're using regex capture-variable $1 unconditionally, and this habit will lead to surprising bugs when the match fails and you get a captured value from different match. You should always wrap uses of $1, $2, and friends inside a conditional, e.g.,

if (/pa(tte)rn/) {
  do_something_with $1;
}

Recommended Fixes

Near the top of your code just after the shebang line, you should immediately add the line

use warnings;

I'd also recommend

use strict;

but that will likely require more work to clean up your code. We're here to help you understand and overcome any issues you find in that worthwhile effort.

Use output_of below to capture a command's output or die with an appropriate diagnostic.

sub output_of {
  my($cmd) = @_;
  my $output = `$cmd`;
  return $output if $? == 0;

  if ($? == -1) {
    die "$0: $cmd failed to execute: $!\n";
  }
  elsif ($? & 127) {
    my $signal = $? & 127;
    my $core = ($? & 128) ? ", core dumped" : "";
    die "$0: $cmd died with signal $signal$core\n";
  }
  else {
    die "$0: $cmd exited with status " . ($? >> 8) . "\n";
  }
}

The section of code from your question becomes

my $output;
$output = output_of 'ec2-run-instances ami-8e1fece7 -k mykey -t t1.micro';
if ($output =~ /INSTANCE\s+(i-\w+)\s/) {
  my $instance_id = $1;

  $output = output_of "ec2-describe-instances $instance_id";
  if ($output =~ /(ec2-(\d+-\d+-\d+-\d+).\S+)/) {
    print "$0: found instance $2\n";  # or whatever
  }
  else {
    die "$0: no ec2-IP in ec2-describe-instances output:\n$output";
  }
}
else {
  die "$0: no INSTANCE in ec2-run-instances output:\n$output";
}

With these changes, your code will supply on the standard error diagnostics for all failure modes in processing output from ec2-run-instances and ec2-describe-instances instead of leaving you to wonder what went wrong.

Answer 2

Is $outp defined or undefined after the:

$outp = `$describe_instances`;

Have you verified that $instance_id is set to what you expect? What are the values of $! and $? after the non-working backtick command? Is anything being written to stderr by the non-working command?

Answer 3

I suspect the problem is this line:

$instance_id = $1;

$1 is a global variable. One should never use $1 et al untested, since it:

Contains the subpattern from the corresponding set of capturing parentheses from the last successful pattern match

(from perldoc perlvar)

In other words, it is not overwritten if there is no match. What you might do is something like this:

die $! unless ( $outp =~ /INSTANCE\s+(i-\w+)\s/ );
$instance_id = $1;

I try never to use $1 untested, and of course there are many ways to make sure it comes from the right place.