26/09/2015 Wannes Van Causbroeck

Test kitchen opsworks workflow

As testing new cookbooks directly in opsworks takes a long time, it makes a lot of sense to test them in vagrant first. There is some documentation available on how to do this, but some of it is out of date, broken or unclear.

I’ve tried to cobble something together that’s quick to set up and works good enough (aka: not 100% identical to opsworks, but close enough)

To mimic an opsworks environment in vagrant, we need several tools:

  • vagrant
  • test kitchen
  • berkshelf
  • chef

Below we describe one way to create and test a new cookbook. There is another way (by using the opsworks provisioner) which we’ll use later. For now, the standard chef_zero provisioner works well enough.
The next setup allows us to test one single cookbook. If you need to test several cookbooks at the same time, things have to change slightly. More about this later.

Creating a project

 

Modify the .kitchen.yml file:

 

To obtain the opsworks attributes, create an opsworks stack + host, log in to that host, and run:

 

This command will provide you the full list of attributes. It’s better to keep only the attributes we need and transfer those over to our environments file. However, the cool thing about opsworks is all hosts can access all data about the stack from these attributes, so it’s worth checking them out in detail to see what’s available.
The example below contains the bare minumum to get things working.

 

Next, if our module has dependencies to other modules, we need to set up our berksfile.

 

You also need to add the dependencies in the metadata.rb file

Test

We didn’t actually specify any real tests in this example (check out the test kitchen manual on how to set this up), but we can provision our test machine with the following command:

 

Once the machine has converged (or not), we can log into the instance to do some manual debugging with:

 

Just as a sidenote: if you want to debug on opsworks, all interesting files are under /opt/aws/opsworks. Even better: you can modifie your cookbooks directly under /opt/aws/opsworks/current/site-cookbooks and run the opsworks-agent-cli with the correct options to rerun chef with your modified cookbooks.

Testing multiple cookbooks at once

There is an important difference between how berksfiles are interpreted by opsworks. You need a different Berksfile and metadata per environment
As a reminder, this is our directory layout:

 

One special remark: Opsworks needs a git repo with only the Berksfile and the two cookbook directories, so take care about how you commit to git!
The main Berksfile for testkitchen is the one under /project/. This calls the Berksfile.in in every cookbook.
I found this code somewhere on the internet, but can’t find the link anymore, so my apologies about not giving credit where credit is due!

 

The sub Berksfiles only contain the needed cookbooks for that specific cookbook. The metadata.rb file is automatically sourced by the cookbook

 

Opsworks needs the following file. Again no metadata is referenced as opsworks will find the necessary files automatically.

 

This is the ‘normal’ Berksfile for cookbook-a. You still need this if you want to test it independently!

 

References

http://pixelcog.com/blog/2014/virtualizing-aws-opsworks-with-vagrant/ (this uses the opsworks provisioner, didn’t get it working yet)
http://enriquecordero.com/programming/opsworks-chef-workflow/
http://docs.aws.amazon.com/opsworks/latest/userguide/opsworks-opsworks-mock.html (outdated)
https://www.youtube.com/watch?v=0sPuAb6nB2o (very interesting test kitchen / berkshelf tutorial)

Leave a Reply

Your email address will not be published. Required fields are marked *