Feeding sed matched block through a command

Question

I have json lines that contain multiple parts per line that look like this:

"SomeDate":"Date(-2156284800000)",

I would like to convert each occurrence in all lines into something more human readable:

"SomeDate":"1901-09-03 00:19:32",

I tried using sed to put the matched block (in this case the timestamp) into the argumentlist of the date command. This fails.

$ echo '"SomeDate":"Date(-2156284800000)",' | \
  sed "s/Date(\([0-9\-]*\)[0-9][0-9][0-9])/$(date -d@\\1 \"+%F %T\")/g"

date: invalid date `@\\1'
"SomeDate":"",

In an attempt to debug this all I added an 'echo' to the date to validate the command it should be running

$ echo '"SomeDate":"Date(-2156284800000)",' | \
  sed "s/Date(\([0-9\-]*\)[0-9][0-9][0-9])/$(echo date -d@\\1 \"+%F %T\")/g"

"SomeDate":"date -d@-2156284800 "+%F %T"",


$ date -d@-2156284800 "+%F %T"

1901-09-03-00:19:32

Why isn't the first command running as expected?

The best guess I have right now is that the subshell is executed first WITHOUT the \1 substitution and then the resulting output is actually used by sed.

How do I achieve what I'm trying to do?

P.S. I'm using CentOS 6.6

note you are opening a subshell to perform the command, so probably the value does not "arrive" there. Also, to execute a command in `sed` you need the `/e`. — fedorqui, Apr 24 '15 at 13:02
If you replace the `\\1` with `-2156284800` you will see that it works, showing that the `\\1` is not matching the string from before, for the reason described by @fedorqui. Fedorqui ,you do not need the `/e` as this works for me ? — , Apr 24 '15 at 13:20
I tried the 'e' option for gnu sed. echo "bla foo bla foo" | sed -e "s@(foo)@echo XX\1XX@e" Desired output: bla XXfooXX bla XXfooXX Actual output: sh: bla: command not found — Niels Basjes, Apr 24 '15 at 13:25
It would be far easier to write a script in a language with a proper JSON parser. — chepner, Apr 24 '15 at 13:39

Lewis Eason · Answer 1 · 2015-04-24T13:57:41.937

0

How about using awk:

echo '"SomeDate":"Date(-2156284800000)",' | awk '{ print gensub(/Date\(([0-9\-]+)\)/, ("date -d@\\1 \"+%F %T\"" |& getline t) ? t : "\\1", "g"); }'

Disclaimer: There's probably a better way to do this, but briefly:

gensub is like gsub but gives you access to the matched groups
Capture the Date(XXX) bit with:
```
/Date\(([0-9\-]+)\)/
```
(which gets the actual epoch in the match group \1)
The second argument is:
```
("date -d@\\1 \"+%F %T\"" |& getline t) ? t : "\\1"
```
which forms the date command, runs it (with getline) and assigns the result to the variable t. getline, bizarely returns 1 on success, so we check for that with the ternary (?:) statement and return the first line of output from that command.
Finally, we tell gensub to be global.

edited Apr 24 '15 at 13:57

answered Apr 24 '15 at 13:52

Lewis Eason

304
2
11

Awk has a known bug and gives the wrong answer for dates before 1970. Your script gives "SomeDate":"1970-01-01 01:00:01", – Niels Basjes Apr 24 '15 at 14:15
Oh, annoying. Either perl, or piping awk into a shell is probably the best route then? – Lewis Eason Apr 24 '15 at 14:19

score 0 · Answer 2 · answered Apr 24 '15 at 14:18

0

The workaround I use right now is via perl:

$ echo fee 4321432431 fie 1429882795 fum | \
       perl -MPOSIX -pe 's/(\d+)/strftime "%F", localtime $1/eg'


fee 2106-12-10 fie 2015-04-24 fum

answered Apr 24 '15 at 14:18

Niels Basjes

10,424
9
50
66

Feeding sed matched block through a command

2 Answers2