7

Team, I cannot see variable substitution while performing xargs command. any hint, how to replace $hostname? on my system hostname is a command and not a variable defined in env. But i can set it to variable also.. 

hostname="$(hostname)"

ls -ltr /var/log/console.* | awk -F 'log/' '{print $2}' | xargs - 
l1 -- sh -c 'aws s3 cp /var/log/"$1" s3://test- 
bucket/dom0/$hostname/' --

output

upload: ../var/log/console.68.log to s3://test- 
bucket/dom0//console.68.log

Please observe dom0// above it should be

expected

dom0/host-123/console.68.log
  • 1
    is `$hostname` defined ? or you want to use the output of the `hostname` command? – nbari Apr 17 '19 at 18:45
  • I am new to xargs. how will my whole expression look with your suggestion? please give exact. –  Apr 17 '19 at 18:53
  • 1
    hostname isn't an environment variable, it's just a variable. If you want it to be an environment variable of your subshells (eg, the shell invoked by xargs via `sh -c`), you should export it. IOW `export hostname` – William Pursell Apr 17 '19 at 19:47

2 Answers2

2

You can safely and easily pass the hostname as a positional parameter in the same way that you pass the filename argument:

hostname="$(hostname)"

ls -ltr /var/log/console.* | awk -F 'log/' '{print $2}' | xargs - 
l1 -- sh -c 'aws s3 cp "/var/log/$2" "s3://test-bucket/dom0/$1/"' -- "$hostname"

This eliminates the fragile injection of variables or filenames into the shell string.

that other guy
  • 116,971
  • 11
  • 170
  • 194
  • @fmaabd Your original command worked by `xargs` running `sh -c 'echo 1 is $1' -- myfilename`, where the argument after the `--` becomes `$1` inside the shell string. I just added an extra argument, so that `$1` is the hostname and `$2` is the filename. – that other guy Apr 17 '19 at 20:12
  • I think i need to understand more: Coz my below did not work for a single file transfer. ls $hostname-var-log.tar.tgz | awk -F '/' '{print $1}' | xargs -l1 -- sh -c 'aws s3 cp $1 s3://test-bucket/dom0/$1/' -- "$hostname" I get below error: The user-provided path np-sjc2-dgx-050 does not exist. xargs: sh: exited with status 255; aborting –  Apr 19 '19 at 22:39
  • It seems like in the current state of this answer, $hostname is passed as $1, but what is $2 in `/var/log/$2` pointing to? – vulcan raven Sep 25 '20 at 14:25
  • @vulcanraven Each input to xargs – that other guy Sep 25 '20 at 17:14
  • Thanks. So this ` | xargs sh -c 'echo "hostname: $1, xarg-input: $2"' -- "$hostname"` is the shorthand of ` | xargs -I{} sh -c 'echo "hostname: $1, xarg-input: $2"' -- "$hostname" {}`. I have always used the latter one; explicit form. :) – vulcan raven Sep 26 '20 at 12:00
0

I had a similar problem in a script I was writing. Based on this answer I tried exporting the variables I needed in my subshell, and that worked for me. Note: I'm using bash and not sh for my xargs subshell, so YMMV.

So for your case, try:

export hostname_="$(hostname)"

ls -ltr /var/log/console.* | awk -F 'log/' '{print $2}' | xargs - 
l1 -- sh -c "aws s3 cp /var/log/$1 s3://test-bucket/dom0/${hostname_}/" --

I added a trailing underscore to differentiate your variable from the command. You can probably get away with leaving out the brace brackets around hostname_. I also tried simplifying the quoting of your subshell command, but I'm not sure that will work; you'll need to test it.

Shi
  • 91
  • 4
  • A question needs to be complete enough that someone can use it without needing to follow links. To leverage the analogy made on [meta.se] in [When is an answer not an answer?](https://meta.stackexchange.com/questions/225370/your-answer-is-in-another-castle-when-is-an-answer-not-an-answer), a sign telling someone where they can get apples is not an apple; likewise, a link to an answer is not an answer itself. Maybe add a demonstration? – Charles Duffy May 15 '23 at 19:49