Decoding JSON and a base64-encoded value in a shell script

Question

I have a JSON and I need to extract a base64-encoded value by particular key and decode it.

JSON has the following structure:

[
  {
    "LockIndex": 0,
    "Key": "Arul/key1",
    "Flags": 0,
    "Value": "MzAKCg==",
    "CreateIndex": 369,
    "ModifyIndex": 554
  }
]

In the above JSON, I need to extract only "Value":"MzAKCg==" and decode the base64-encoded "MzAKCg==" value. I would like to perform this using shell scripting.

Please assist.

score 27 · Answer 1 · answered Oct 29 '18 at 19:31

27

jq has recently added support for base64 encoding and decoding

https://stedolan.github.io/jq/manual/#Formatstringsandescaping

@base64:

The input is converted to base64 as specified by RFC 4648.

@base64d:

The inverse of @base64, input is decoded as specified by RFC 4648. Note: If the decoded string is not UTF-8, the results are undefined.

For your data, the command would be

jq -r 'map(.Value | @base64d)' < file.json

https://github.com/stedolan/jq/issues/47

It's not released yet, but you can install the latest development version to use it.

brew reinstall --HEAD jq

Once the next version of jq is released then you can switch back to the latest stable version.

answered Oct 29 '18 at 19:31

mpkuth

6,994
2
27
44

6

Note that if you have an object whose values are base64-encoded (e.g., a Kubernetes secret), you can use `.data | map_values(@base64d)`. – weberc2 Feb 08 '21 at 18:01
this ALMOST solved my problem... but (by the doc you linked and tests here) `@base64d` only works for UTF-8 stuff, and I need to deal with UTF-16 content. Damn, almost. Thanks anyway! – João Ciocca Apr 14 '23 at 17:33

score 10 · Accepted Answer · answered Dec 01 '16 at 14:47

10

Using jq and base64:

jq -r '.[].Value' < file.json | base64 --decode

answered Dec 01 '16 at 14:47

Ruslan Osmanov

20,486
7
46
60

7

This works if you're trying to decode one value at a time, but if you have an object of values (e.g., `{"secret": {"DB_USERNAME": "", "DB_PASSWORD": ""}}`) then this doesn't work very well. Note that this isn't a contrived case, but rather the way Kubernetes secrets are structured. – weberc2 Feb 08 '21 at 17:49

score 2 · Answer 3 · answered Nov 02 '21 at 11:35

2

This worked for me: curl --silent localhost:8500/v1/kv/key1 | jq -r '.[0].Value | @base64d'

answered Nov 02 '21 at 11:35

Bogdan Daragan

106
4

score 1 · Answer 4 · edited Aug 31 '21 at 14:36

1

if this JSON will have always the same structure, you can use cut -d and later decode the value, for example:

$echo "Value": "MzAKCg==" | cut -d ":" -f 2 | base64 -D

30

edited Aug 31 '21 at 14:36

StephenKing

36,187
11
83
112

answered Jul 03 '18 at 16:22

letakeda

155
3
12

score 1 · Answer 5 · answered Dec 14 '21 at 15:38

1

use built-in in jq base64 decoder:

echo '{"val" : "MzAKCg=="}' | jq -r '.val |=@base64d'

yields:

{ "val": "30\n\n" }

side note: jq is simply a file symbolic for jq-win64.exe

answered Dec 14 '21 at 15:38

visitant

11
1

score 1 · Answer 6 · answered Aug 19 '22 at 08:49

1

if you want to keep the origin struct, you could use like jq -r '.|=map(.key|=@base64d)

answered Aug 19 '22 at 08:49

Cong Wu

372
2
16

score 0 · Answer 7 · answered Sep 03 '21 at 18:53

0

With xidel:

xidel -s input.json -e '$json()/binary-to-string(base64Binary(Value))'

or

xidel -s input.json -e 'binary-to-string(base64Binary($json//Value))'

answered Sep 03 '21 at 18:53

Reino

3,203
1
13
21

Decoding JSON and a base64-encoded value in a shell script

7 Answers7