Bash: command substitution when reading from a file

Question

Not sure if I am being dumb here but I am not able to figure this out.

If I put this in bash script:

var="`date '+%a %b %d'`*.*INFO Ended execution of script"
echo $var

The output that I get is:

Mon Sep 27*.*INFO Ended execution of script

However, If I put the same content in a file:

"`date '+%a %b %d'`*.*INFO Ended execution of script"

And then write a small bash script to read the content of this file

while read -e line
do
        echo "$line"
done < file

When I execute this script I get

"`date '+%a %b %d'`*.*INFO Ended execution of script"

How do i get the command substitution to work when I read the content from a file?

Short of using `eval` (not recommended), you don't. Text read by `read` is not input to the shell, and so isn't subject to shell processing like command substitutions. — chepner, Sep 27 '21 at 19:01
See [BashFAQ #50](http://mywiki.wooledge.org/BashFAQ/050) (and, in terms of why-`eval`-is-a-bad-idea, [BashFAQ #48](http://mywiki.wooledge.org/BashFAQ/048)). — Charles Duffy, Sep 27 '21 at 19:03
...there are very good reasons this doesn't work: if it was otherwise it would be basically impossible to write code that securely handles untrusted data in shell languages. You don't expect data with Python code embedded in it to run that code when you tell it to print that data; why should shell be different? — Charles Duffy, Sep 27 '21 at 19:03
Also, command substitutions are _slow_. If you need a high-performance approach to handle a lot of data, they're the wrong tool for the job (and bash 4.3 and later has newer/better/faster built-in tools for quickly formatting dates). — Charles Duffy, Sep 27 '21 at 19:06

score 3 · Accepted Answer · answered Sep 27 '21 at 19:19

3

To be clear, this is an extremely bad idea, and is not something you should ever actually do. However, it would look like:

while IFS= read -r line; do
  eval "echo $line"           ## DANGEROUS: Do not do this!
done

In terms of why it's a bad idea:

It's innately insecure. BashFAQ #48 goes into more details on that, but in short, if your data can run the date command, it can run any other command as well.
It's slow. $(date) starts a new copy of /usr/bin/date each time it's run. Using printf -v date '%(%a %b %d)T' -1 stores the date formatted the way you want in $date far faster.

answered Sep 27 '21 at 19:19

Charles Duffy

What we are trying to do is write a shell script that will monitor logs to verify whether all scheduled jobs have run for the day. The idea was to store patterns in a property file , read it from a bash script and verify the presence of those patterns in log files. Each line in the property file may have a different format so this pattern has to be dynamically determined. what would be the best approach that would suggest then to solve this? – Nishant Shrivastava Sep 27 '21 at 19:46
Anyways, many thanks for the suggestion Charles Duffy. Based on your feedback, i have decided to make some design changes to solve this in a more secure way . Once again, thanks for not only the solution but also the associated downside of using this approach. – Nishant Shrivastava Sep 27 '21 at 19:54

1 Answers1