Hot deploying app builds to Vagrant

Question

Please note: although I mention Gradle/Groovy in this question, this is strictly about Vagrant usage, and could just as easily apply to any other language/build system.

I am new to Vagrant and am trying to figure out how Vagrantfiles, when pushed to source control, interact with that repo's build.

Say I have the following repo, built by Gradle:

src/
    main/
        groovy/
            <lots of groovy source>
    test/
        groovy/
            <lots of groovy test source>
build.gradle
settings.gradle

Now let's say that, without Vagrant, I would normally clone this repo and run gradle clean build to run unit tests on the code, and ultimately, package it into a running program (executable JAR). Then a gradle run starts the app.

Well, in a lot of GitHub repos I see Vagrantfiles committed. So obviously this is to make it easy to spin up a VM and run the respective app inside of. I'm trying to figure out the typical "flow" of incorporating a Vagrantfile into my repo above such that developers can:

1. Clone the repo
2. Build with Gradle
3. Somehow, deploy to the Vagrant box
4. Test the running app instance (running inside the box)
5. Tweak the code } #4, #5 and #6 quick/rapid dev-test-tweak cycles utilizing hot re-deploys
6. Re-test

Take a look at this Vagrantfile for a CAS server (chosen at random). I think this is where the magic happens, in terms of deploying and restarting the server on the box, but I checked the Vagrant docs for shell.inline and nothing concrete came up.

So I ask: How do I "integrate" my Vagrantfile with my build, such that the build produces a deployed, running app? Further, what do my run/deploy-test-code-redeploy cycles look like with Vagrant boxes (what I call "hot deploying")?

Answer 1

I'm going to explain how I would achieve what you are looking to do, this might not be the official way to do it, so anyone with more Vagrant experience should please provide pointers on where it can be improved. I've been working with Vagrant for about 6 months now. You say you are new to Vagrant, so I'll be as complete as I can, even though some parts you have probably already mastered, but might be useful to other users.

tl;dr; Skip to Creating the Vagrantfile section if you are already familiar with how the Vagrantfile works

Let's start with what the example Vagrantfile is doing.

config.vm.box = "puppetlabs/centos-6.5-64-puppet"

This is telling Vagrant to pull the box on which everything else is built. You can find a lot of official and community contributed boxes on the Vagrant Cloud. This one being from puppetlabs, based on CentOS 6.5 64-bit with Puppet already installed.

config.vm.network "forwarded_port", guest: 8080, host: 8088

This is telling Vagrant to forward port 8088 on your host to 8080 on the Vagrant box. So accessing http://127.0.0.1:8088 on your host will access port 8080 on the guest.

config.vm.provision :shell do |shell|

Provisioning is the process of setting up the Vagrant box when it is ran for the first time. Provisioning will only run once for a new VM, unless forced. See the provisioning section of the Basic Usage Vagrant Docs. This one specifically is executing a bunch of shell commands when it is being provisioned.

shell.inline = "command;
                command;"

Shell inline is sending these commands, seperated by the semicolon, to the box. This is to automate the commands as if you were typing them in an SSH session yourself. The Vagrant Docs on Shell Provisioning has some more advanced uses where you can define an actual file to be executed. Note: Your shell script should not try to execute tools that have not yet been installed. (ex. Running a Python script where Python is not available yet).

config.vm.provision "puppet" do |puppet|

I can't comment much on the Puppet section since I'm not a Puppet user (yet, probably). This is setting some Puppet values.

shell.inline = "cd /vagrant && mvn clean package;
                  sudo cp target/cas.war /srv/tomcat/cas/webapps/;
                  sudo /sbin/service tomcat-cas restart"

This is also executing shell commands. Basically changing directory, cleaning, copying cas.war to the webapps directory and then restarting the service. More on the /vagrant shared folder later. Now we have enough to start building our own Vagrant file on. I'm going to keep the sample simple to make it generic.

Creating the Vagrantfile

You will most likely want to build on a Vagrant box that already matches your requirements, but for now, let's not do that. You can however find a lot of already created boxes on the Vagrant Cloud. I'm going to show you how to get a (very) simple Python app running using Flask.

1. Pick your favourite distribution from the available Vagrant boxes. I'm going to use ubuntu/trusty64 since I use it on a daily basis. All command should be easily translated to other distributions.
2. Create a new directory for your project and open a shell / console window in it.
3. Using the console, initialize your Vagrant box vagrant init ubuntu/trusty64. This will create a base Vagrantfile for you to work from.
4. Open your Vagrantfile and uncomment config.vm.network "forwarded_port", guest: 5000, host: 8080. We want port 8080 to take us to port 5000 on the guest machine. Tip: For your project, it will be wise to choose a port that is most likely not already in use to avoid clashing with other apps. 8080 might be bad choice, 8089 will be better.

5. Let's add some scripts to execute on provisioning. Since Python is shipped with Ubuntu (and most other I know) we don't need to install Python, but we do need pip (a Python Package manager) and Flask.

   config.vm.provision :shell do |shell|
       shell.inline = "cd /vagrant;
           sudo apt-get -y install python-pip;
           sudo pip install Flask;"
   end
   

   This will change the directory to the Vagrant share. Install pip using Ubuntu's package manager, then install Flask using pip. The -y flag is to automatically have apt-get install without prompting for a yes/no question. You might need to run Vagrant up --provision a couple of times to get all your commands 100% correct.

   Note on the Vagrant share: The Vagrant share is a directory that is synced between the host and guest machine and will be available under /vagrant. It includes all the files and directories in your project directory (where your Vagrantfile resides).

6. We now have all the tools we need to run our app. I've created an incredibly simple Flask app as a sample. Download it to your project directory from this Gist and name it app.py

7. Now we'll be able to run the Python app located in your project directory. I like to keep the install sections and running sections seperate. So add another section that starts the app. Note: The & makes the app fork to the background so the vagrant up can complete. You'll probably want to do something more fancy than this with your app.

   config.vm.provision :shell do |shell|
     shell.inline = "/vagrant/app.py &"
   end
   

8. Finally we can start everything and have Vagrant do its magic. In your console, in your project directory (where your Vagrant file is located). Run vagrant up.

9. Moment of truth. Open your browser (on the host) and browse to http://127.0.0.1:8080/. You should see Hello Vagrant Provisioned World!.

That takes care of the provisioning of your app. Automatically, from a Vagrantfile, that you can commit with your project code.

Now to get to your initial steps, and how this fits in.

Integrating into your development workflow

I'm listing your steps, integrated with Vagrant.

1. Clone the repo This step stays the same, with the exception of a Vagrantfile included in your repo with provisioning for any required libraries and tools required for your project.
2. Run vagrant up in the project directory. This will automatically create and provision the box and share your project with the box.
3. You can build the project in the Vagrant provisioning steps, but if you are actively developing the application, you might not want to do that.
4. Test the running app instance (running inside the box) - Simply SSH into the box, enter the `/vagrant' directory and run your app.
5. Tweak the code } #4, #5 and #6 quick/rapid dev-test-tweak cycles utilizing hot re-deploys Since your project is shared live between the host and the guest. you can simply stop the app (if running) and run the app again on the guest. No copying needed.
6. Re-test

This will give you quick developent cycles while keeping the environment the same. A new developer can simply clone and vagrant up to get going on the project without worrying about the environment and whatnot.

Continuous Integration is a vast topic. You can still apply the practises to your repo, etc. But I'd skip the CI deployment process while developing. I make use of Jenkins and Capistrano for my CI deployments, but it is too heavy weight for development. In production I won't use Vagrant since it will double-virtualize your VM already (unless you run bare metal). For production I'll make use of Docker and Fig.

I hope this explains how to integrate Vagrant into your flow for your project, please do comment if anything needs clarification. I believe the sample should word perfectly, since that is the goal of using Vagrant.