3

Say I want to convert a date in timestamp to another format through the date command. In console I would say date -d@<timestamp> but I happen to want to do this to many fields in a text file.

Using the e to execute in sed (sed (GNU sed) 4.2.2, actually) I am saying:

$ echo 1449158360 | sed -r 's#.*([0-9]{10}).*#date -d@\1 "+%Y";#e'
2015

It works, nice!

Now I created a dummy file myfile:

my timestamp is 1449158360 but also I wonder what date was 1359199960.

Which I would like to have replaced to the same but having the relative year of the timestamps:

my timestamp is 2015 but also I wonder what date was 2013.

However, if I try to run the same command as above it fails:

$ sed -r 's#([0-9]{10})#date -d@"\1" "+%Y";#e' myfile
sh: my: command not found
sh: but: command not found

Because sed interprets the first words as something to execute.

Obviously it works if I just fetch these data and nothing else:

$ sed -r 's#.*([0-9]{10}).*#date -d@"\1" "+%Y";#ge' myfile
2015

So I wonder: what should I do to call date against captured groups in sed and replace text with it, considering it is surrounded by other text that have to remain untouched?

anubhava
  • 761,203
  • 64
  • 569
  • 643
fedorqui
  • 275,237
  • 103
  • 548
  • 598
  • Possible duplicate of [How to convert all unix dates in a file?](http://stackoverflow.com/questions/33776948/how-to-convert-all-unix-dates-in-a-file) – Toby Speight Dec 03 '15 at 16:52
  • 1
    Not really, @TobySpeight since here I am asking why it is not working the way I am trying to do it. – fedorqui Dec 03 '15 at 16:53
  • Did you read the answers there? Specifically, [my comments on sed `s///e`](/a/33781069/4850040) - I recommend Perl rather than sed for this job. – Toby Speight Dec 03 '15 at 16:56
  • 1
    @TobySpeight I read them, there is even a deleted answer from me there : ) I see the point here is about what you mention in your answer, _Note that /e causes the command substitution to replace the entire pattern space, so you may need to make use of hold space to retain the text before and after the substitution. That's left as an exercise for the reader_ It is this last part that I am unable to perform. I will edit later to focus on this specific problem instead. – fedorqui Dec 03 '15 at 17:00

2 Answers2

6

e switch in sed substitute applies sh -c to unmatched text as well as evident from this command:

echo 'a 1449158360' | sed -r 's#([0-9]{10})#date -d@\1 "+%Y";#e'
sh: a: command not found

So even though we are matching only 1449158360 but sh -c is being run on a 1449158360.

Due to absence of non-greedy and lookaheads regex in sed this workaround regex might appear crazy but this is how you can run it for multiple matching input from file as in your question:

sed -r 's#(([^0-9][0-9]{0,9})*)(\b[0-9]{10}\b)(([0-9]{0,9}[^0-9])*)#printf "%s%s%s" "\1" $(date -d@\3 "+%Y") "\4";#ge' file

Basically we are matching <before>10-digits<after> in this regex.

Output:

my timestamp is 2015 but also I wonder what date was 2013.

To clarify the regex used I have created this demo.

By no means this is a generic solution to e mode issue, treat it as a regex based workaround.

anubhava
  • 761,203
  • 64
  • 569
  • 643
  • 1
    Wow this is beautiful, many many thanks! I learnt two things: first, how to execute a command in `sed` effectively. Then, how to work with nested captured groups (I initially wondered where was `\2`, then I saw `\1` can be multiple cases of `\2`). Many thanks! – fedorqui Dec 04 '15 at 10:10
  • 1
    Yes unfortunately because of limitation of `sed` regex i.e no lazy quantifier, no lookahead and no non-capturing group, this regex became a bit complex. Glad that you liked it. It was a very interesting problem for me. – anubhava Dec 04 '15 at 10:19
  • 1
    Thank you, thanks to this I managed to create a handy command to output `~/zsh_history` with human readable dates as `sed -E 's/(([^0-9][0-9]{0,9})*)(\b[0-9]{10}\b)(([0-9]{0,9}[^0-9])*)/printf "%s%s%s" "\1" "$(date -ud @\3)" "\4"/ge' ~/.zsh_history | less` – tiagovrtr Jun 30 '21 at 15:53
0

As you said, with the e flag sed is getting all the sentence as something to execute.

A turnaround could be something like that.

Change myfile to:

echo "my timestamp is 1449158360 but also I wonder what date was 1359199960."

then:

sed -r 's#([0-9]{10})#\`date -d@\1 "+%Y"\`#ge' myfile

Hope that's help

djangoliv
  • 1,698
  • 14
  • 26