How to store/get json to/from aws parameter store

Question

I use Livy REST API to submit Spark app.

{
  “file”: <application-jar>,
  “className”: “<main-class>”,
  “args”: my_args,
  “conf”: my_conf
}

my_args = [args1, args2, ...]
my_conf = {'foo1': 'bar1', 'foo2': 'bar2'...}

I want my_conf (json secrets) to store in AWS SSM parameter store and to use it from parameter store when my python script using Livy submits spark app. How to store and get(get in python the same json) my_conf to/from parameter store?

Answer 1

import json
import boto3

my_conf = {'foo1': 'bar1', 'foo2': 'bar2'}
boto3.client('ssm').put_parameter(Name='MyParameter', Value=json.dumps(my_conf), Type='String')
my_conf = json.loads(boto3.client('ssm').get_parameter(Name='MyParameter'))

...assuming you've set up the proper permissions to allow you to make these calls.

See

• https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.get_parameter
• https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.put_parameter

Answer 2

I have used this utility with success in the past. The advantage is it builds a config that's human readable and editable in ssm should you so choose.

https://github.com/b-b3rn4rd/json2ssm