5

Problem

We use java WAR files and keep config files in s3 buckets. Our environments: DEV,QA, Stage, and PROD each have their own config files and s3 buckets. If I add a new field, such as "Polling_RATE=5000", it must be manually added to each env because these config files also store passwords so they can not be tied to the application or kept inside Github. Not every engineer has access to each env so you must remember to inform the upper level engineers (DEVOPS) before the prod deployment date to add the new field for the application to work. Its a really messy process currently.

Question

Is there a utility or architectural design pattern meant to deal with this? How do you "version control" sensitive configuration fields that you can not store within github?

Usman Mutawakil
  • 4,993
  • 9
  • 43
  • 80

2 Answers2

2

Recognizable problem.

Usually config fields with sensitive information like passwords change a lot less often than non-sensitive configuration fields. A possible solution is to split the config in two parts:

  1. Config that's environment-specific but doesn't contain sensitive information. I would advise you to keep these files together with your source code and if possible, generate the files and automatically upload then to your configuration store (S3 in your case) at build time. They must be versioned and tied to the version of your application.
  2. Config that contains sensitive information. Looking at the question, not all team members are allowed to read/write this information. You could store these in S3 with specific access rights so that only authorized members can access them. You would need a mechanism to join the files back together at deployment, or change the app to read from different config files.
    However, this will only solve part of your problem. The ops guys will still need to perform changes when sensitive config keys change. Whether this is acceptable depends on how often sensitive config keys change.

An alternative to S3 could be to run a private Git repository (AWS's CodecCommit, for example). You'd have better version control and easier access for the devs to perform changes, since you're already using Git. You'll still have to fix the split access rights between dev and ops, or let that go (since DevOps is about trust and cooperation, that might be a good idea). You could apply a similar pattern here as I described above.

Another solution could be to move the configuration of sensitive values from property files to the system configuration. When you already use a provisioning system like Puppet or Chef, this will feel natural for the ops guys. Or set all sensitive values like passwords as environment variables and have the app read it as system properties.

Hope this helps!

Bert Jan Schrijver
  • 1,521
  • 10
  • 16
0

We have been using dynamodb for keeping config values. The advantage with this approach is that the values are easily readable from console and validated. Also another advantage is that we periodically check the values from dynamodb so if any value needs to be changed we just change it and the app automatically picks the new value instead of starting it again. Sensitive values are stored encrypted using KMS keys and only the ec2 role that is running the application has right to decrypt using that Key. We enhanced the Netflix archiaus project to fit our needs. May be you can check that out.

Rohit
  • 842
  • 6
  • 8
  • Thanks Rohit. So does this mean you have one dynamodb instance holding the configuration for each env? – Usman Mutawakil Feb 29 '16 at 14:58
  • Yes. 1 table per environment. And a Jenkins job to load the data into this table from csv which is maintained as part of git repo. – Rohit Feb 29 '16 at 15:12
  • If I add new properties to the DEV dynamodb instance. Must I manually remember to update the 3 other dynamodb instances or does this single jenkins job cover all envs? – Usman Mutawakil Feb 29 '16 at 15:52
  • 1
    It is part of the code and is promoted with the code. When you deploy the code to that env you also update the properties for that env. We have different property file for every env, but have validation test to ensure that all env files have the same set of properties. – Rohit Feb 29 '16 at 16:44