How to use sed to convert epoch to human readable date time

Question

I tried use sed with /e option to call date -d @ to convert epoch to human readable date time but it seems that sed works only if the input line is very simple

I prepared a bash file to simply convert epoch in my format and copied it in usr/local/bin that it's in my $PATH

#!/bin/bash
date +'%Y-%m-%d %T %Z' -d @$1

and I can demonstrate that it works fine:

$ echo 1616720904 | sed 's/\(.*\)/epoch-to-date \1/e'
2021-03-26 02:08:24 CET

but:

1. if you input a complex string like usually, you have from a log file, it doesn't work:

$ echo "timestamp:1616720904" | sed 's/\(timestamp\:\)\([0-9]*\)/`epoch-to-date \2`/e'
sh: 1: 2021-03-26: not found

as you can see 2021-03-26 is part of the converted date time and seems passed to the bash again sh: .... not found

2. if you have more text in the input string it pass all text to the bash:

$ echo "version:123,timestamp:1616720904" | sed 's/\(timestamp\:\)\([0-9]*\)/`epoch-to-date \2`/e'
sh: 1: version:123,2021-03-26: not found

The output I would get is:

version:123,timestamp:2021-03-26 02:08:24 CET

Answer 1

My advise is to extract the epoch from your strings then call epoch-to-date on the result:

for s in 1616720904 timestamp:1616720904 version:123,timestamp:1616720904
do
  epoch-to-date "$(echo "$s" | sed 's/version:[0-9]\+,//; s/timestamp://')"
done

Answer 2

1. The reason why this failed is not the complexity of the input string, but simply that in contrast to the example which works fine you put the command in ` `, which causes the output of it to be interpreted as a command. Just leave out the ` `.

2. The same applies here. Besides that, to remove more text in the input string, cater for the text before the timestamp by including it in the search pattern:

   <<<version:123,timestamp:1616720904 sed 's/.*timestamp:\([0-9]*\)/epoch-to-date \1/e'
   

   (I also removed the unnecessary capturing of unwanted text.)
   The same is needed if there's more text after the time.

But how to print out the source input string with the converted epoch?

Indeed then we need the ` `:

<<<version:123,timestamp:1616720904 sed 's/\(.*timestamp:\)\([0-9]*\)/echo \1`epoch-to-date \2`/e'

Answer 3

Do not use e option with sed - most of the time it's very confusing. In your case, just:

<<<version:123,timestamp:1616720904 sed 's/.*timestamp:\([0-9]*\)/\1/' | xargs epoch-to-date 

Answer 4

Thanks to everybody I did epoch-to-date utility that I want to share:

#!/bin/bash
if [ -f $1 ]; then
    #echo is_file
    #looks for 10 digits number within the line and try to convert it
    cat $1 | sed 's/\(.*\)\([0-9]\{10\}\)\(.*\)/echo \1`epoch-to-date \2`\3/e'
else
    #echo is_epoch
    date +'%Y-%m-%d %T %Z' -d @$1
fi

Copy this file in your home directory, make it executable and link it from /usr/local/bin or other directory in your $PATH

sudo chmod +x epoch-to-date
cd /usr/local/bin
sudo ln -s /home/<your_name>/epoch-to-date

Usage:

$ epoch-to-date 1616720904
2021-03-26 02:08:24 CET

or convert a whole log file at once:

$ cat file.log
EP_REQ/GZ200101 {"cmd":"Reboot","mode":"HW","hw_ver":"1.02","fw_ver":"1.2.3","cfg_ver":"0","timestamp":1616720904}

$ epoch-to-date file.log
EP_REQ/GZ200101 {cmd:Reboot,mode:HW,hw_ver:1.02,fw_ver:1.2.3,cfg_ver:0,timestamp:2021-03-26 02:08:24 CET}

$ cat file.log | epoch-to-date
EP_REQ/GZ200101 {cmd:Reboot,mode:HW,hw_ver:1.02,fw_ver:1.2.3,cfg_ver:0,timestamp:2021-03-26 02:08:24 CET}

Answer 5

This might work for you (GNU sed & bash):

echo "etd(){ date +'%F %T %Z' -d@\$1; }" >funlib
val="version:123,timestamp:1616720904"
<<<"$val" sed -E 's/(.*:)(.*)/bash -c '\''. funlib;echo "\1$(etd \2)"'\''/e'

Write a function etd to a file called funlib.

Assign the string version:123,timestamp:1616720904 to variable val.

Using a here-string <<< pattern match the two parts of the $val and then using the e flag of the substitution command, evoke two commands within bash to source the funlib file and then call the function within to get the result of the etd function using the second back reference from the LHS of the substitution command as the only parameter and echo the result out.

N.B. The default shell of the sed evaluation is hard coded to /bin/sh this is usually a symbolic reference to dash, hence bash needs to be explicitly called if needed.