Dan Stutzman

Embed just a cookbook in your Ruby project, not a whole kitchen

Published 2014-01-13

TL;DR To embed a Chef cookbook in your Ruby web project, run berks cookbook cookbook from your project's top-level directory. Setup Vagrant so everyone can easily deploy. But for production, point to the cookbook from a Berkfile in a kitchen outside your project, with the git or path option.

If you're a Ruby web developer and you handle deployment yourself (instead of having a sysadmin or Heroku do it for you), you can imagine the benefit of automating deployment. This post assumes that you've decided on Chef because of its popularity with the Ruby community, and now you're planning to write a Chef cookbook and distribute it with your Ruby project, so that future maintainers can easily deploy with your cookbook.

Unfortunately, the Chef documentation assumes you have a single "chef-repo" in version control, which contains cookbooks for all your projects, as well as info you wouldn't want to make public, such as passwords. So it's not obvious how your chef-repo and your project's repo should relate to each other. Given the enormous flexibility of Chef and Ruby and community tool contributions, there must be a way to supply a cookbook along with your project -- but how?

(For the rest of this post, I'll assume you're using chef-solo with a kitchen directory created by knife solo init, instead of chef-server with a chef-repo, but the two situations are similar.)

Solution #1 (bad): embed your project inside your cookbook

Recommended by Mischa Taylor's post, but it's probably intended as a toy example.

Approach Start with an empty Berks cookbook: berks cookbook myface. Later, add your source code to templates (e.g. templates/default/index.php.erb for a PHP app) and refer to it with template from your recipe.

Advantage It's straightforward to see how your Chef recipe finds your project's source code files; they're right under templates or files.

Disadvantage Requires turning your project into a Chef cookbook, burying its source files under templates and files. A developer may not be able to run your app without deploying it with Chef.

Solution #2 (okay): store your project's cookbook in a totally separate repo

Recommended by Jamie Winsor's post (we don't see where the dropbox.com artifacts come from, so I assume it's from another repo)

Approach Create a separate Git repository for your Chef cookbook. For example, if your project were named ort you'd have a separate repo named ort-cookbook. You and every developer will then use librarian-chef or Berksfile to refer to that cookbook from their kitchen.

Advantage This option is the easiest to understand, and corresponds best to the Berkshelf and librarian-chef docs.

Disadvantage Requires two Git repos. An outside developer could be completely unaware of the second Git repo or neglect to keep the two in sync.

Solution #3 (better): embed an entire kitchen inside your project

Recommended by Frank Hoffsümmer's blog post and Dan Ivovich's slides

Approach Run kitchen solo init to create a "my_kitchen" directory inside your project. Now "my_kitchen" will contain a cookbooks and a site_cookbooks directory. According to the Knife solo docs, your custom cookbook(s) should go in site_cookbooks since the cookbooks dir will be managed by Berkshelf or librarian-chef.

Advantage Developers unfamiliar with Chef are given a working example, just by running vagrant up.

Disadvantages

• Developers may confuse the cookbooks and the site_cookbooks directory.
• Non-custom cookbooks will be mixed together with your custom cookbook in the cookbooks directory.
• Developers may be tempted to deploy to production from this directory, so they'll either commit deployment info (IP addresses, node info, users, passwords, etc.) to Git, or keep them out of Git altogether by using .gitignore.
• Commits to this info might trigger unwanted continuous integration runs.
• It's easy to overlook the hidden .chef directory.

Solution #4 (best): embed cookbook inside your project; point to it from kitchen outside project

Recommended by me :-)

Approach

1. Install Berkshelf and relevant Vagrant plugins:

   gem install berkshelf --no-ri --no-rdoc
             vagrant plugin install vagrant-berkshelf
             vagrant plugin install vagrant-omnibus

2. Create a cookbook directory named cookbook containing a Vagrantfile and Berksfile:

   berks cookbook cookbook
             cd cookbook
             bundle install
             

3. Fill out metadata.rb with information for your cookbook. Cookbook dependencies will go here too, for example: depends 'apt', '~> 2.3.4'
4. Tweak the Vagrantfile as needed, for example:
   • Set the config.vm.hostname
   • Change the config.vm.box to "precise32" and config.vm.box_url to "http://files.vagrantup.com/precise32.box" or whatever your prefer, to avoid being dependent on the Berkshelf box.
   • Add config.omnibus.chef_version = :latest since the precise32 box won't have chef auto-installed.
   • Delete config.ssh.max_tries and config.ssh.timeout, which are deprecated settings.
   • Add config.vm.boot_timeout = 120
   • Add port forwarding if you prefer
   • Change the recipe in the runlist from cookbook to whatever you named the recipe in metadata.rb
5. Start the VM with vagrant up. It will automatically provision itself using chef-solo using the recipe at recipes/default.rb
6. Test your app at http://33.33.33.10/ or http://localhost:8080/ or whatever you have set up.
7. Iterate to improve your cookbook by editing recipes/default.rb and re-running vagrant provision. Repeat as needed.
8. When you're ready to deploy to staging or production, cd to your kitchen in another directory outside this project, and in your kitchen's Berksfile, refer to your project's cookbook by adding one of the following lines:
   • cookbook 'myproject', git: 'https://github.com/myname/myproject.git', rel: 'cookbook'
   • cookbook 'myproject', path: '/Users/myname/myproject/cookbook'

Advantages

• Developers unfamiliar with Chef are given a working example, just by running vagrant up.
• Non-custom cookbooks are hidden at ~/.berkshelf/cookbooks, where developers are unlikely to edit them.
• The custom cookbook's directories (e.g. recipes, files, etc.) are directly visible under cookbook.
• The cookbook is stored in the same repo as the project, so the two can stay synchronized.

Disadvantage Well, it is a little confusing to have two Berksfiles (one in your project, one in your kitchen) and three Gemfiles (project dir, cookbook dir, kitchen).

Agree? Disagree? I'm new to Chef and would love to hear your insight in the comments below.