Sed: execute alternative command when output from previous one is empty

Question

I wanted to modify dig command to automatically apply a reverse lookup to any A record I receive in output.

Therefore, I've created the following function:

dt ()
{
    remove=$(echo $@ |  sed 's^https*^^' | sed 's^[,/*:]^^g' );
    dig any +trace +nocl +nodnssec $remove | sed "s/\($remove.*A\s*\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\)\)/dig -x \2 +short | xargs echo \"\1 ->\"/e"
}

With this code I have the following output (only A record part is shown so to avoid the question getting bigger):

domain.com.      1200    A        198.54.115.174 -> server224-3.web-hosting.com.

However, now I also need to make a whois lookup using the IP I receive from dig output, but only in case dig -x \2 +short doesn't give any result (stackoverflow.com can be a good example of a domain with A records that do not have PTR).

I tried something like this to check exit code of regular host command (since dig implies that output is successful even if it's empty) and execute proper command depending on the result:

dig any +trace +nocl +nodnssec "$remove" | sed -e "s/\($remove.*A\s*\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\)\)/echo -n \"\1 -> \"; host \2 | grep -q ''; if [ ${PIPESTATUS[0]} -eq 0 ]; then dig -x \2 +short; else whois \2 | grep 'network:Network-Name:';fi; /e"

But it seems that sed somehow affects the value of ${PIPESTATUS[0]} array.

I wanted to do these modifications in sed because I needed something that will print lines on the go. If I use variables and modify the output from them, the function will work slower, at least visually.

Maybe awk can be of use here, but I am not sure how I should write the code using this command.

Is there a way around this problem? Can sed be used for this purpose or should I use some other tool? Thanks.

You have: `remove=$(echo $@ | sed 's^https*^^' | sed 's^[,/*:]^^g' );` — you should consider: `remove=$(echo "$@" | sed -e 's^https*^^' -e 's^[,/*:]^^g' );`. The change uses one `sed` invocation instead of two; that's uncontroversial. Using `$@` unquoted is usually a danger sign, but it does flatten out white space and newlines, so it isn't absolutely wrong. Unquoted, there's no difference between `$@` and `$*`; quoted, there is. It isn't absolutely clear what's best there, but `"$@"` is usually the correct way to quote arguments. —— None of this has anything much to do with your problem. — Jonathan Leffler, Dec 18 '18 at 23:19
I just made the mistake of trying to look at the far right end of your sed scripts. They're too long and unreadable to be interesting to me, I'm sorry to say. At least, not while I'm at work and need to be doing other stuff as soon as the compilation is done. — Jonathan Leffler, Dec 18 '18 at 23:22
@JonathanLeffler thank you for your recommendations. And sorry for not using "-E" parameter for sed (which would make the regex a bit smaller and easier to look at). Just wanted to make sure I leave out any possible edits that could add more mistakes to the line, since it was written ~9 months ago (when I didn't know about ERE option), so I might also need to review the regex itself. — Игорь Иванов, Dec 19 '18 at 00:01
Also, I will certainly add "-e" flag to optimize the line for "remove" variable per your recommendation. I am also trying to figure out how to make my code more readable. Most probably I should at least break lines by "\" or something. — Игорь Иванов, Dec 19 '18 at 00:02
I think you're writing a shell script (fragment) with the `sed` scripts. I wonder if you'd do better having an actual second script that you invoke with a few (one, two, …) appropriate arguments. If it's unreadable, it's unmaintainable. It may be correct, but it is hard to tell. I seem to be unusual these days, but I write lots of scripts, and most of them aren't very big, at least when they start out. I have over 600 programs (scripts and executables) in my `$HOME/bin` directory. — Jonathan Leffler, Dec 19 '18 at 00:06
Or, if you don't like shell scripts, consider making another function which takes the appropriate arguments and runs the code. — Jonathan Leffler, Dec 19 '18 at 00:20

score 0 · Answer 1 · answered Dec 19 '18 at 00:07

The good old' bash gives you variety of tools grep, awk, xargs, while read and so on. Using sed with e command with inside checking PIPESTATUS and executing xargs.... is just unreadable and very long too read. It's not clear to me what do you want to do, as too much happens on one line.

The dig command results zero status on success - you can check it's output if it has zero length when it "fails".

Consider the following script:

dt () {
    remove=$(
        <<<$1 \
        sed 's#^http[s]?://\([^/]*\).*#\1#'
    )
        dig any +trace +nocl +nodnssec "$remove" |
        tr -s '\t' |
        awk -v "remove=${remove}." '{
                if ($1 == remove && $3 == "A") {
                        print
                }
        }' |
        while IFS= read -r line; do
                IFS=$'\t' read -r _ _ _ ip _ <<<"$line"
                if ! dig_result=$(dig -x "$ip" +short); then
                        echo "Dig error" >&2
                        exit 1
                fi
                if [ -z "$dig_result" ]; then
                        printf "%s -> no-result-from-dig\n" "$line"
                else
                        printf "%s -> %s\n" "$line" "$dig_result"
                fi
        done
}

First dig is run. Notice the "$remove" is quoted.
Just a precaution - I squeze the tabulators in the output.
Then I filter - the first column should have "$remove". with a dot and the third column should be an "A"
Then for each line (as there are meny)
I get the ip address (ok, maybe ip=$(<<<"$line" cut -f4) would be cleaner and simpler).
I get the result from the dig -x using the ip address
If the result is empty I print "no-result-from-dig" if it not, I print the result.

Running:

dt stackoverflow.com
dt google.pl

Outputs:

stackoverflow.com.      300     A       151.101.65.69 -> no-result-from-dig
stackoverflow.com.      300     A       151.101.193.69 -> no-result-from-dig
stackoverflow.com.      300     A       151.101.129.69 -> no-result-from-dig
stackoverflow.com.      300     A       151.101.1.69 -> no-result-from-dig
google.pl.      300     A       216.58.215.67 -> waw02s16-in-f3.1e100.net.

The reason your command did not work is that $PIPESTATUS was quoted inside " quotes thus expanded before running the shell. You should escape the $ so that's \$PIPESTATUS or pass it inside nonexpading single quotes " ... "'$PIPESTATUS'" ... ".

Thank you so much! That's the solution I've been looking for! I will need to adjust some things, but now I can see some good examples on how to develop the function further. — Игорь Иванов, Dec 20 '18 at 17:36

Sed: execute alternative command when output from previous one is empty

1 Answers1