0

I want to dump a sha checksum (here using sha1sum, I know it is not to be used for crypto, do not worry this is fine for my need) to disk from a bash script. So far, I am able to get it dumped (without anything extra...). But I do not manage to dump it in binary format, instead I am only getting a plaintext hex dump:

$ echo "bla" | sha1sum | awk '{print $1}' | head -c-1 > test
$ ls -lrth test
-rw-r--r-- 1 jrlab jrlab 40 mars   2 15:02 test
$ xxd test
00000000: 3034 3735 3935 6430 6661 6539 3732 6662  047595d0fae972fb
00000010: 6564 3063 3531 6234 6134 3163 3761 3334  ed0c51b4a41c7a34
00000020: 3965 3063 3437 6262                      9e0c47bb

For example here, if I am right, the output of sha1 is truly 20 bytes long, which takes 40 chars to represent in hex printout (i.e. 40 bytes with encoding the hex printout in ASCII, and this is the reason why xxd can transcript all bytes of the file as chars 0-f), and this is what is present on my file. How can I change this so that the size of test is 20 bytes on disk, i.e. truly dumped in binary format?

Sorry if I missed an easy way to do this, I have been googling (probably the wrong question) for quite some time without finding a clear answer.

Zorglub29
  • 6,979
  • 6
  • 20
  • 37

1 Answers1

2

Found it! After a bit more digging in forums / man pages, xxd is able to solve it for me (found it here: https://gist.github.com/dstebila/6194f90097c43c091dd2 ):

# Convert hex to binary using xxd's reverser in plain hexdump style
xxd -r -ps

So xxd is able to take a file filled with ASCII hexdump, and convert it to a binary content:

$ echo "48454C4C4F" > test.hex
$ xxd test.hex
00000000: 3438 3435 3443 3443 3446 0a              48454C4C4F.
$ xxd -r -ps test.hex > test.bin
$ xxd test.bin
00000000: 4845 4c4c 4f                             HELLO
$ ls -lrth test.hex
-rw-r--r-- 1 jrlab jrlab 11 mars   2 15:44 test.hex
$ ls -lrth test.bin
-rw-r--r-- 1 jrlab jrlab 5 mars   2 15:44 test.bin
Zorglub29
  • 6,979
  • 6
  • 20
  • 37
  • Very useful! Though why `-s`? Without an offset (or `seek`) according to the man page, it really isn't doing anything. – David C. Rankin Mar 02 '20 at 18:16
  • @DavidC.Rankin From the manual page of xxd, it looks like -ps is different from -p combined with -s. Not with my computer now, do you want to try? From https://www.systutorials.com/docs/linux/man/1-xxd/ I get -p | -ps | -postscript | -plain output in postscript continuous hexdump style. Also known as plain hexdump style. So looks like xxd break the -- conventions in args passing :( – Zorglub29 Mar 02 '20 at 19:25
  • Hah, my bad, I thought that was two short-form options together, `-p -s` as `-ps` - no worries. – David C. Rankin Mar 02 '20 at 19:39
  • I know, no worries, this is what I thought first, very confusing / non Unix convention there :( . – Zorglub29 Mar 02 '20 at 19:44
  • Btw, any idea where / how to report / complain about this unusual notation? Would be good to ask for a possibility to use --ps even if --ps is kept, just to let people express intent better ^^ . – Zorglub29 Mar 02 '20 at 19:47
  • I don't know if it will do much good. It doesn't look like the man page has been updated since `"August 1996"`, and at that point there wasn't a great deal of standardization in one-off utilities. But it does list the author's e-mail, at least as of 1997 `:)` – David C. Rankin Mar 02 '20 at 20:38