8

I really like the approach of a 12factor app, which you are kinda forced into, when you deploy an application to Heroku. For this question I'm particularly interested in setting environment variables for configuration, like one would do on Heroku.

As far as I can tell, there's no way to change the ENV for one or multiple instances within the EC2 console (though it's seems to be possible to set 5 ENV vars when using elastic beanstalk). Therefore my next bet on an Ubuntu based system would be to use /etc/environment, /etc/profile, ~/.profile or just the export command to set ENV variables.

Is this the correct approach or am I missing something?

And if so, is there a best practice on how to do it? I guess I could use something like Capistrano or Fabric, get a list of servers from the AWS api, connect to all of them and change the mentioned files/call export. Though 12factor is pretty well known, I couldn't find any blog post describing how to handle the ENV for a non-trivial amount of instances on EC2. And I don't want to implement such a thing, if somebody already did it very well and I just missed something.

Note: I want a solution without using elastic beanstalk and I don't care about git push deployment or any other Heroku-like feature, this is solely related to app configuration.

Any hints appreciated, thanks!

Daniel Torres
  • 749
  • 1
  • 7
  • 17
  • Just found this, which might be interesting for folks finding this post via google: [capistrano-twelvefactor](https://github.com/yanted/capistrano-twelvefactor). – Daniel Torres Mar 01 '14 at 20:53

3 Answers3

3

Good question. There are many ways you can approach your deployment/environment setup.

One thing to keep in mind is that with Heroku (or Elastic Beanstalk for that matter) you only push the code. Their service takes care of the scalability factor and replication of your services across their infrastructure (once you push the code). If you are using fabric (or capistrano) you are using a push model too, but you have to take care of all the scalability/replication/fault tolerance of your application.

Having said that, if you are using EC2, in my opinion it's better if you leverage AMIs, Autoscale and Cloudformation for your deployments. This is the beauty of elasticity and Virtualization in that you can think of resources as ephemeral. You can still use fabric/capistrano to automate the AMI builds (I use Ansible) and configure environment variables, packages, etc. Then you can define a Cloudformation stack (with a JSON file) and in it you can add an autoscaling group with your prebaked AMI.

Another way of deploying your app is to simply use the AWS Opsworks service. It's pretty comprehensive and it has a lot of options but it may not be for everybody since some people may want a bit more flexibility.

If you want to go 'pull' model you can use Puppet, Chef or CFEngine. In this case you have a master policy server somewhere in the cloud (Puppetmaster, Chef Server or Policy Server). When a server gets spun up, an agent (Puppet agent, Chef Client, Cfengine agent) connects to its master to pick up its policy and then executes it. The policy may contains all the packages and environment variables that you need for your application to function. Again, it's a different model. This model scales pretty well but it depends on how many agents the master can handle and how you stagger the connections from the agents to the master. You can load balance multiple masters too if you want to scale to thousands of servers or you can just simply use multiple masters. From experience, if you want something really "fast" Cfengine works pretty good, there's a good blog comparing the speed of Puppet and CFengine here: http://www.blogcompiler.com/2012/09/30/scalability-of-cfengine-and-puppet-2/

You can also go "push" completely with tools like fabric, Ansible, Capistrano. However, you are constrained by how much a single server (or laptop) can handle multiple connections to thousands of servers that its trying to push to. This is also constrained by network bandwidth, but hey you can get creative and stagger your push updates and perhaps use multiple servers to push. Again it works and it's a different model so it depends which direction you want to go.

Hope this helps.

Rico
  • 58,485
  • 12
  • 111
  • 141
  • Interesting answer, thanks. It does not actually answer my question, but maybe my question was wrong. Given your answer, it seems more like the approach of trying to get a Heroku-like ENV-based setup into EC2 doesn't make sense. Maybe it does indeed make more sense to use something like Opsworks and handle the config completely in there. I was just very obsessed with the idea of not having config data like database urls in any git repo, as it's the case with Heroku. But after all, separating the config from the app (12factor) could also mean the config lives in it's own repo... – Daniel Torres Feb 12 '14 at 21:10
  • Well, it does make sense if you want more flexibility and are willing to spend time and resources in replicating a Heroku/Elastic Beanstalk type of setup. There are many tools that you can use to do that `fabric` is one of them for example. If you don't have the resources and time Opsworks is something that can help you get there faster (with a little less flexibility perhaps). – Rico Feb 12 '14 at 22:16
0

If you dont need beanstalk, you can look at AWS Opsworks (http://aws.amazon.com/opsworks/). Ideal for Web worker kind of deployment scenerios. You can pass any variable from outside the code here (even Chef recipies)

Sony Kadavan
  • 3,982
  • 2
  • 19
  • 26
0

It's might be late but they what we are doing.

We have python script that take env var in Json and send that to as post data to another python script that convert those vars to ymal file.

After that we use Jenkins pipline groovy using multibranch. Jenkins do all the build and then code deploy copies those env vars to ec2 instanced running in autoscaling. Off course we are doing some manapulation from yaml to simple text file so code deploy can paste it on /etc/envoirments

Ali Ahsan
  • 3
  • 4