Google Deployment Manager stored runtime-config variables appear corrupted

Question

I'm having trouble with encrypted base64 encoded values I'm using in Google Deployment Manager via runtimeconfig.v1beta1.config resource declarations.

After I perform the deployment, the value that I stored using Deployment Manager appear to be quite different to what I retrieve using gcloud beta runtime-configs. As a result, I can't decrypt the value.

First I encrypted and base64 encoded some secret text:

$ echo "secret"|gcloud kms encrypt --key my-crypto-key \
  --keyring my-keyring --location australia-southeast1 \
  --plaintext-file - --ciphertext-file - | base64 -w0

CiQAsOSNmVXBs2ayUjRePnE5+Oi5dUPuVvjn6UKKUXgxMTA56koSMABDkVUGnXlocFgdUEsQ5qLCF3PVIz5zit+ZCSXjSvNzEAO5XRv6WBRkxBJMjVcheg==

Which I then store in a deployment manager YAML file:

resources:
- name: my-config
  type: runtimeconfig.v1beta1.config
  properties:
    config: my-config
    description: "A demo configuration"

- name: dummy-secret
  type: runtimeconfig.v1beta1.variable
  properties:
    parent: $(ref.my-config.name)
    variable: 'dummy/secret'
    value: "CiQAsOSNmVXBs2ayUjRePnE5+Oi5dUPuVvjn6UKKUXgxMTA56koSMABDkVUGnXlocFgdUEsQ5qLCF3PVIz5zit+ZCSXjSvNzEAO5XRv6WBRkxBJMjVcheg=="

Then I create the deployment (which completes without errors or warnings):

$ gcloud deployment-manager deployments create my-config \
  --config my-config.yaml

But when I try extracting the variable value, it is completely different from what I stored:

$ gcloud beta runtime-config configs variables \
  get-value 'dummy/secret' --config-name my-config|base64 -w0
CiQAPz8/P1U/P2Y/UjRePnE5Pz8/dUM/Vj8/P0I/UXgxMTA5P0oSMABDP1UGP3locFgdUEsQPz8/F3M/Iz5zPz8/CSU/Sj9zEAM/XRs/WBRkPxJMP1cheg==

This is repeatable / reproducible and I haven't a clue what I'm doing wrong. I don't have this problem using gcloud beta runtime-config variables set followed by get-value.

Tim Dierks · Answer 1 · 2018-07-12T17:15:40.237

1

Looking at the decoded base64 binary of your content, we notice that all the bytes with values >= 0x80 have been changed to 0x3F, ASCII '?'. We suspect you're passing the binary data through the shell or some other pipe which isn't binary-clean.

Corrupted value:

dierks@dierks:~$ base64 -d | hexdump -C
CiQAPz8/P1U/P2Y/UjRePnE5Pz8/dUM/Vj8/P0I/UXgxMTA5P0oSMABDP1UGP3locFgdUEsQPz8/F3M/Iz5zPz8/CSU/Sj9zEAM/XRs/WBRkPxJMP1cheg==
00000000  0a 24 00 3f 3f 3f 3f 55  3f 3f 66 3f 52 34 5e 3e  |.$.????U??f?R4^>|
00000010  71 39 3f 3f 3f 75 43 3f  56 3f 3f 3f 42 3f 51 78  |q9???uC?V???B?Qx|
00000020  31 31 30 39 3f 4a 12 30  00 43 3f 55 06 3f 79 68  |1109?J.0.C?U.?yh|
00000030  70 58 1d 50 4b 10 3f 3f  3f 17 73 3f 23 3e 73 3f  |pX.PK.???.s?#>s?|
00000040  3f 3f 09 25 3f 4a 3f 73  10 03 3f 5d 1b 3f 58 14  |??.%?J?s..?].?X.|
00000050  64 3f 12 4c 3f 57 21 7a                           |d?.L?W!z|
00000058

Original value:

dierks@dierks:~$ base64 -d | hexdump -C
CiQAsOSNmVXBs2ayUjRePnE5+Oi5dUPuVvjn6UKKUXgxMTA56koSMABDkVUGnXlocFgdUEsQ5qLCF3PVIz5zit+ZCSXjSvNzEAO5XRv6WBRkxBJMjVcheg==
00000000  0a 24 00 b0 e4 8d 99 55  c1 b3 66 b2 52 34 5e 3e  |.$.....U..f.R4^>|
00000010  71 39 f8 e8 b9 75 43 ee  56 f8 e7 e9 42 8a 51 78  |q9...uC.V...B.Qx|
00000020  31 31 30 39 ea 4a 12 30  00 43 91 55 06 9d 79 68  |1109.J.0.C.U..yh|
00000030  70 58 1d 50 4b 10 e6 a2  c2 17 73 d5 23 3e 73 8a  |pX.PK.....s.#>s.|
00000040  df 99 09 25 e3 4a f3 73  10 03 b9 5d 1b fa 58 14  |...%.J.s...]..X.|
00000050  64 c4 12 4c 8d 57 21 7a                           |d..L.W!z|

edited Jul 12 '18 at 17:15

answered Jul 12 '18 at 14:41

Tim Dierks

2,168
15
28

Hi Tim, The gcloud encrypt command outputs binary in my first codeblock above, but piping doesn't corrupt it there, as I can take the base64 encoded value it produced, store directly using gcloud: `gcloud beta runtime-config configs variables set 'dummy/secret' "CiQA[SNIP]" --config-name my-config` And it can still be decrypted: `gcloud beta runtime-config configs variables get-value 'dummy/secret' --config-name my-config|base64 -d | gcloud kms decrypt --key my-crypto-key --keyring my-keyring .. --plaintext-file - --ciphertext-file - secret` – Poo Bah Jul 12 '18 at 21:34
OK. I don't know anything about deployment manager (I works on KMS), but can see that the value you're getting out of KMS looks OK. I don't know whether the value is getting corrupted by high-bit octets getting replaced with '?' on their way into the deployment or on their way out. GIven that it's base64 encoded in the YAML file, encoding of that file isn't a problem. If you can set a value with gcloud and get it back, then gcloud and value extraction aren't likely to be the problem. So maybe the YAML parser? I'll ping someone on the team. – Tim Dierks Jul 13 '18 at 01:21
Workaround is to modify the YAML so the dummy-secret property 'value' is changed to 'text', which results in the base64 encoded encrypted secret being stored as a string in runtime-config. – Poo Bah Jul 15 '18 at 11:06
1

Have raised a ticket with Google: https://issuetracker.google.com/issues/111407084 – Poo Bah Jul 15 '18 at 11:07
Is a bug in gcloud. Per above ticket. – Poo Bah Jul 18 '18 at 09:44

Google Deployment Manager stored runtime-config variables appear corrupted

1 Answers1