Matthew Loberg Software Engineer

Advance Jekyll

This is part two in a three part series on static site generators.

  1. Why Static Site Generators
  2. Getting Starting With Jekyll
  3. Advance Jekyll (You are here)

In the previous post I talked about Jekyll, a popular static site generator written in Ruby. In fact, it’s what this site has been using for a while. During that time, I’ve picked up some tips and tricks that I want to share with you today.

The things you find in here are not best practices, absolute truths, or even the best way to do things. With development, there’s a thousand different ways to do something, these just happen to be what I found works.

In This Post

Deploy To S3

GitHub Pages offers a free solution for hosting a Jekyll site on a platform you probably already use. This works wonderfully for the majority of sites out there. However, because it is a hosted solution, there’s limitations to it. You can’t run custom plugins and the build process has to be completely done through Jekyll. The power of Jekyll is really unlocked when you deploy to another platform that you have control over.

Because Jekyll outputs static files, the options for where you can deploy your site are endless. You could manually upload the files to a web server you own or even automate that process with a tool like rsync.

This site is hosted on Amazon S3. It was a product I was already using and I don’t have to worry about security, updates, or any of the other things you have to with a normal web server. They have static website hosting built in, but it can be a bit confusing to get setup. Instead, I recommend using a tool called s3_website. Not only does it manage S3 static hosting configuration, it can upload your site for you (and only the files that changed), use AWS CloudFront to distribute your website, manage HTTP cache control and gzipping, and much more. Obviously you don’t want to commit your AWS access or secret keys, so either keep the file local, or commit an example copy of it without those values.

Manage Your Dependencies

As you dive deeper into customizing and streamlining your Jekyll site, you’ll find yourself using lots of different tools. Using a dependency management tool like Bundler can help easy that and provide consistency between any different environments you may be using.

To get you started, here is a starting Gemfile for Jekyll.

source 'https://rubygems.org'

gem 'jekyll'

You can also target a specific version of Jekyll.

source 'https://rubygems.org'

gem 'jekyll', '~> 3.3'

Make sure you also commit your Gemfile.lock file to manage specific versions of dependencies.

Stop Excluding Files

We’ve added two more files to our directory that we probably don’t want to be a part of our final site. If we add more tools like Gulp or Yarn, we would need to add more files to exclude from our site. Jekyll allows you to define the source of your site. By default this is the current directory (.), we can change that with the source option. Either on the command line or even better, our _config.yml file.

source: site

This directory can contain only files that Jekyll uses to generate the site and you won’t have to worry about extra files getting into your build.

Plugins

Now that we have all the boring stuff out of the way, we can start getting to the cool parts. Arguably one of the most powerful features of Jekyll is it’s plugin system. There are plugins out there to generate sitemaps, RSS feeds, add SEO tags, and more. There’s even support for some plugins on GitHub Pages.

There are a couple different ways to install plugins in Jekyll. The first is to list it in your _config.yml.

gems:
  - jekyll-sitemap

If you’re using Bundler, you can use the jekyll_plugins group and those will automatically be registered with Jekyll.

# Gemfile

group :jekyll_plugins do
  gem 'jekyll-sitemap'
end

You can find out all about Jekyll plugins at their documentation.

Custom Liquid Tags

Liquid is a pretty powerful templating language. With Jekyll we can build on top of that and add custom tags. This is done by as a Jekyll plugin, all we need is a little Ruby.

As an example, I want a tag that strips out all extra whitespace. To do this, I create a file called _plugins/spaceless.rb.

module Jekyll
  class Spaceless < Liquid::Block

    def initialize(tag_name, text, tokens)
      super
    end

    def render(context)
      text = super.to_s
      lines = text.split("\n").map { |line| line.strip }.reject! { |line| line.empty? }
      lines.join("\n") if lines
    end

  end
end

Liquid::Template.register_tag('spaceless', Jekyll::Spaceless)

Now we have access to a spaceless tag in our site.

{% spaceless %}
     This won't have whitespace before it
{% endspaceless %}

Custom Filters

You can also add Liquid filters. Filters are Ruby modules that export all their methods to Liquid.

module Jekyll
  module MyFilters
    def asset_url(input)
      "http://my-cdn.example.com/assets/#{input}"
    end
  end
end

Liquid::Template.register_filter(Jekyll::MyFilters)

Filters are applied via pipe ({{ profile.jpg | asset_url }}).

Assets

I remember when Jekyll didn’t support Sass or CoffeeScript out of the box, luckily that’s all changed and adding Sass to your site is easy. Concatenating CSS/JS, optimizing images, and adding in third-party assets aren’t as easy. If Jekyll out of the box works for you, that’s great. If you need something a little bit more advance, read on. I’m going to go over two different options. Gulp or Jekyll Assets.

Gulp

Gulp is a build system for JavaScript. I’ve personally used it for other projects and I love it. I’m not a front-end developer by any means, but it’s really easy to use. Unfortunately, there’s a million different ways to configure your build. If you’re looking for a place to start, I would check out generator-jekyllized.

I was never able to get a build pipeline that I liked in Gulp, so I don’t have much insight here, but there are tons of sites out there that have done this.

Jekyll Assets

I was never able to get a build pipeline in Gulp that I liked, so I decided to go with a pure Ruby option of Jekyll 3 Assets. If you’re not comfortable with NodeJS or Gulp, but need something better than what Jekyll provides out of the box, this might be a good solution for you. Jekyll Assets is an asset pipeline for Jekyll 3 using Sprockets, the same system Rails uses. With the support for Sprockets, I was able to switch over to Jekyll Assets from my old Gulp build in about 10 minutes.

Testing

We have this badass site. It builds all our assets for us, we’ve customized it with some third-party plugins and even a few of our own. It’s perfect, how do we make sure it stays that way? We can test it!

What can we test? First we can test that everything with Jekyll is ok. Let’s run the doctor command.

$ jekyll doctor

We should also test that all the links in our site are going to resolve to a working page. To do this, we use the [html-proofer][html-proofer] gem. This will scan all links in HTML pages and make sure they return a real page. I’ve had this catch links that were broken on old posts.

$ jekyll build
$ htmlproof [site-dest]

For most sites, this is probably sufficient. Let’s say you have a site that uses a lot of custom plugins or logic in your templates that you want to test. We could use a tool like Selenium or a lighter tool like PhantomJS. I used CasperJS for this site. It offers some really useful tools specific for testing. Here is an example test:

casper.test.begin('Homepage', 7, function suite(test) {
  casper.start('http://localhost:4000/', function() {
    test.assertTitle("Matthew Loberg · Software Engineer");
    test.assertNotVisible('#sidebar', 'Sidebar is not visible');
    test.assertElementCount('.post', 5, 'There should be 5 posts on the first page.');
    this.click('.older');
  }).then(function() {
    test.assertUrlMatch(/page2/, "We are on the second page of posts.");
    test.assertElementCount('.post', 5, 'There should be 5 posts on the second page.');
    this.click('.sidebar-toggle');
  }).then(function() {
    test.assertVisible('#sidebar', 'Sidebar should be visible after clicking hamburger.');
    test.assertElementCount('#sidebar .social ul li', 2);
  }).run(function() {
    test.done();
  });
});

You can check out the full test for this site on GitHub.

Additionally you could do some CSS regression testing using a tool like BackstopJS, Wraith, or PhantomCSS.

Automated Deployments

One of the things I really missed about GitHub Pages was the ability to push a change to my site and have it automatically built. With s3_website, I need to remember to build and deploy my site. If I’m away from home, I can’t make a change with GitHub’s interface and have it automatically build.

Or can I?

With a tool such as TravisCI it is possible to have automatic deploys on push. First let’s create a .travis.yml that just builds and tests our site.

language: ruby
sudo: false
rvm:
  - 2.1

cache:
  directories:
    - vendor

env:
  global:
    - TZ=America/Chicago # Makes sure posts are generated with the right timestamp
    - NOKOGIRI_USE_SYSTEM_LIBRARIES=true # speeds up installation of html-proofer
    - JEKYLL_ENV=production

install: bundle install --jobs=3 --retry=3 --path vendor --standalone --deployment

script:
  - bundle exec jekyll build
  - bundle exec htmlproof build

Here we’re using a Ruby 2.1 Travis container to build our site. I’ve set a custom install script that installs gems to vendor and then caches that directory to speed up the installation process. For our test script, we build the site and then run [html-proofer][html-proofer] against the build.

Travis has an after_script option that will run after a successful build. We don’t want to expose our AWS access and secret key, so we need to add them as encrypted variables. Then change our s3_website.yml to load those from the environment instead.

s3_id: <%= ENV['S3_ID'] %>
s3_secret: <%= ENV['S3_SECRET'] %>
s3_bucket: <%= ENV['S3_BUCKET'] %>
cloudfront_distribution_id: <%= ENV['CF_ID'] %>

Finally, we only want to run this on the master branch. Travis gives an option to only run on certain branches.

language: ruby
sudo: false
rvm:
  - 2.1

cache:
  directories:
    - vendor

env:
  global:
    - TZ=America/Chicago # Makes sure posts are generated with the right timestamp
    - NOKOGIRI_USE_SYSTEM_LIBRARIES=true # speeds up installation of html-proofer
    - JEKYLL_ENV=production
    - secure: xxx
    - secure: xxx
    - secure: xxx
    - secure: xxx

install: bundle install --jobs=3 --retry=3 --path vendor --standalone --deployment

script:
  - bundle exec jekyll build
  - bundle exec htmlproof build

after_success: bundle exec s3_website push

branches:
  only:
    - master

If we make a change to our site using GitHub’s interface, it should automatically build and push our site. Huzzah!

Wrapping Up

This is only scratching the surface of what Jekyll and static site builders are capable of. If you have any useful tips or tricks you’ve learned with Jekyll, please share them in the comments.

Getting Started With Jekyll

This is part two in a three part series on static site generators.

  1. Why Static Site Generators
  2. Getting Starting With Jekyll (You are here)
  3. Advance Jekyll

In the first post I talked about what static sites are and a little bit about static site generators. I talked a lot about Jekyll, which what we are going to be talking about today.

Jekyll is a static site generator written in Ruby. It uses Markdown (or Textile), and Liquid to generate a static site. It’s also blog-aware, which means there’s a lot of cool features around blog posts such as permalinks and categories. In fact this blog post you are reading right now, was generated using Jekyll.

Installation

To use Jekyll, you’ll need Ruby, RubyGems, and a *nix system. If you are using Jekyll 3 you need Ruby v2 or above. If you are using Jekyll 2, you will need Ruby 1.9.3 or greater, NodeJS, and Python 2.7. Because Jekyll is distributed as a Gem, installation is really easy.

$ [sudo] gem install jekyll

Creating a New Site

To look at what’s in a Jekyll site, we are going to create a new site using the jekyll new command. This will setup some boilerplate code for a Jekyll site.

$ jekyll new myblog
$ cd myblog
$ jekyll serve

This will start a development server at http://localhost:4000.

Directory Structure

Let’s take a look at the directory structure.

.
|___ _config.yml
|___ _includes
| |___ footer.html
| |___ head.html
| |___ header.html
| |___ icon-github.html
| |___ icon-github.svg
| |___ icon-twitter.html
| |___ icon-twitter.svg
|___ _layouts
| |___ default.html
| |___ page.html
| |___ post.html
|___ _posts
| |___ 2016-01-27-welcome-to-jekyll.markdown
|___ _sass
| |___ _base.scss
| |___ _layout.scss
| |___ _syntax-highlighting.scss
|___ about.md
|___ css
| |___ main.scss
|___ feed.xml
|___ index.html

Configuration

The _config.yml file is the configuration file for Jekyll. One of the cool things about Jekyll is that none of these files are required. Even though this file holds the configuration for the site, there are enough defaults that it can build the site without this file. In addition you can add custom settings that will become global site variables.

Includes

The _includes directory holds any partials that can be included in your layouts or pages. To do this, use the include Liquid tag. For example: {% include footer.html %} will include _includes/footer.html.

Layouts

The _layouts directory holds all the page templates for the site. Layouts are chosen by the YAML Front Matter of a page. To inject the page content, the {{ content }} Liquid tag is used.

Posts

Remember how I said Jekyll is blog-aware? The _posts directory contains all posts for the site and will generate permalinks based on the file name. The default will use the format /:categories/:year/:month/:day/:title.html, but you can customize that using _config.yml.

Assets

Your site won’t just be HTML pages, you’ll have CSS and possibly some JavaScript. You could use just normal CSS or Jekyll has support for both Sass and CoffeeScript. In the site generated for us, it has some Sass files. First there is a _sass directory, which is our Sass partials directory. The we have css/main.scss. If you look at this file, you’ll notice it has empty YAML Front Matter. In order to process through Jekyll, files must include a YAML Front Matter block. You can read more about assets at Jekyll’s docs.

YAML Front Matter

Before we talk about how to add a page to Jekyll, let’s talk about YAML Front Matter. YAML Front Matter is a block of YAML that is used for building the page. It can contain information such as the layout to use, the page title, permalink, or any other custom settings or page variables. Any page that contains Front Matter, will be processed by Jekyll. Otherwise the file will just be copied over during build. Front Matter must be the first thing in a file. It must start and end with three-dashed lines. Here is an example:

---
layout: default
title: About
permalink: /about/
---
<!-- page content -->

Content

Now that we’ve covered what Front Matter is, let’s talk about creating some content. First, let’s take a look at index.html. This is an HTML file, but because it contains Front Matter, it will be processed by Jekyll and use the default template. The HTML file can contain just HTML with Front Matter, but it can also contain Liquid tags. Liquid is a templating language that feels similar to Twig or Jinja. For example, if we want to list out all our posts, we can do that using this block of code:

<ul>
    {% for post in site.posts %}
        <li><a href="{{ post.url }}">{{ post.title }}</a></li>
    {% endfor %}
</ul>

That’s pretty cool, but sometimes writing HTML for every page is lame. Luckily we don’t have to as Jekyll supports Markdown and Textile. This will take Markdown (or Textile if you so choose) and then convert it into an HTML file. How awesome is that? You can write your blog posts in Markdown!

With content, you aren’t just limited to HTML or Markdown files. Any file that contains Front Matter can use Liquid and be processed by Jekyll during the build. For example, in the site Jekyll gave us, it has a feed.xml file that creates an RSS feed from all the posts. You could also create a JSON file that contains information about all your blog posts to do searching through JavaScript.

Customization

I hope you’re getting a sense of how cool Jekyll is and all the stuff you can do with it, but we’ve barely scratched the surface. There’s drafts, collections, data files, pagination, and variables that allow you to do some pretty cool stuff. If that’s not enough for you, you can add plugins or even write your own to really show how powerful Jekyll can be.

I’m not going to dig into this stuff in this post. If you are interested, part three of this series will be about some more advance Jekyll features and tricks, otherwise the Jekyll docs has a ton of information.

Deploying to GitHub Pages

Now that you’ve got a functioning Jekyll site that you can view with jekyll serve, how do you get that deployed for the world to see? There’s a lot of options here because it is just static files. You can drop them on any web server. You can also create this advance script that will deploy your site around the globe using Amazon Web Services every time you run git push (which I’ll be talking about in the next post). Or you can keep it simple (and free) and deploy to GitHub Pages. We’re going to be talking about deploying an account/organization site, but you can also have a site for a project as well.

Setting up a site through GitHub Pages is easy. All you have to do is create a repository called username.github.io, push your code to the repository, and wait for it to build (the first time can be slow).

This will host your site at http://username.github.io/. If you want to use your own domain name, worry not, it’s really easy. All you need to do is add a file called CNAME that contains your custom domain name. This should not include the http: or any slashes (mlo.io not http://mlo.io). Commit and push this file. Then you’ll need to add record to your DNS provider. If you are using a subdomain, you can use a CNAME record. This is the recommended option as it gives you the benefit of their Content Delivery Network and any changes to the IP of the GitHub servers (which they’ve done) won’t affect your site. If you can’t use a CNAME record, you can use an A record. To get the current IP address and more information, please visit their help page.

While GitHub Pages is awesome, free, and easy, there are some limitations and potential downfalls to the service. The first is that you can’t run any custom plugins, you can only run plugins listed on [this page][ghpage-plugins]. One thing you’ll notice, is that at the time of writing GitHub Pages is only running Jekyll 2.4.0. While this shouldn’t be an issue for most sites, it may cause some inconsistencies between what’s built on your machine and what GitHub Pages will deploy. Finally, the biggest issue for me with GitHub Pages in the past was downtime. A couple years ago, my site would go down for hours at a time. I’m not sure if this is still an issue or if they have improved their architecture to prevent it from happening, but it was the reason why I moved my site to AWS.

Build Something Awesome

With all this new-found knowledge of Jekyll, I hope you build some cool sites or migrate over your blog to Jekyll. Maybe you’re still not 100% sold on the idea of a static site. In the next post, I’m going to go through some more advance Jekyll features and hopefully show you that Jekyll is a lot more powerful than you think.

Update 2/1/16: GitHub Pages just upgraded to Jekyll 3.0 along with a few other changes to the service. You can view their blog post for more information.

Why Static Site Generators

This is a 3 part series about static site generators and Jekyll in specific.

  1. Why Static Site Generators (You are here)
  2. Getting Starting With Jekyll
  3. Advance Jekyll

What the hell is a static site generator? First we need to understand how most web sites work. Most web sites you visit are dynamic sites, which means the content is generated at request time. This could include calls to a database, a third party service, or other processes that could take time to processes.

A static site is just static content (HTML, CSS, JavaScript). Nothing is generated, no database calls are made. Static content is simply returned. That means these sites perform better because serving static content is a lot easier on the web server.

A static site generator builds those static files from different templates or scripts and can make managing a static site a lot easier than having to update each HTML page if you make a change to some layout element.

In fact, this site you are looking at is a static site. There is no database, just static files. I’m using a tool call Jekyll, that I’ll be talking about more in-depth in later posts.

Why Should I Care?

As a web developer, my personal site has gone through many different systems. For a while it was a WordPress site, at a different time it was a custom built application. A couple years ago I made the switch to Jekyll and I haven’t looked back. I’m going to give you just 5 reasons why static sites and Jekyll is awesome, although I think there are many more benefits than this.

1. No Servers

I think my favorite thing about how I’ve hosted my site since switching to Jekyll is that I haven’t need to manage or maintain a server. I used Github Pages for a while, and now I’m using Amazon S3 and CloudFront. No having to worry about setting up a server and making sure it’s up or Apache is working.

You could host your static site on your servers, but there are plenty of options out there, where you don’t need to.

2. No Upgrades

I think one of the worst things when I had a WordPress site are the constant updates. With Jekyll I don’t have any software on the server that I have to worry about upgrading, and I don’t even need to upgrade Jekyll unless I want to.

3. No Security Worries

WordPress is always getting updates because of security vulnerabilities being discovered. Or what about a custom application, who knows what security issues could be lurking in there. Because Jekyll is just static files, there’s no scripts that are ran on the server.

This also goes for server software. Because I’m not running my own server, I don’t have to worry about making sure Apache or nginx or whatever other software is update to date.

4. No Downtime

How many times have you seen an article featured on Reddit or Hacker News that you visit and it’s unavailable because it wasn’t able to support the influx of traffic. Or if something in your code breaks, or some configuration on your server gets messed up, there goes your site. With static sites, handling that influx of traffic is a lot easier and Apache can handle it a lot better than dynamic sites where the issue is usually with the database.

If you use a service like Github Pages* or host your site on Amazon CloudFront or some other CDN, you don’t even have to worry about making sure Apache or nginx can handle that.

* There have been cases in the past where I have experienced downtime with Github Pages, which is why I moved to S3 and CloudFront. Downtime usually only last a couple minutes at most.

5. No Costs

Assuming you’re not counting your domain registration and you are using Github Pages, hosting a site that uses Jekyll is free. Even with my site now that uses S3, CloudFront, and Route 53, I only pay $2.50/month and that’s including all the other things I have on S3.

Potential Downfalls

But Matt, I can’t switch my site over to a static site. My site needs to have…

1. Comments

Scroll down to the bottom of this page, what is that you see? Comments?!? How can it be? Thanks to a really awesome service called Disqus, you can have comments using just JavaScript.

2. Real Time Content

An old version of my site used to pull in my Twitter feed and also my Last.fm tracks. When I switched over to Jekyll I was able to pull in this data with JavaScript. I’m not sure how viable this option is as your API keys may need to be in JavaScript or you could hit API limits. Do you really need this information pulled in on your site? It is nice, but with all the other benefits of static sites, you could probably get by without this.

3. Admin UI

With WordPress it was nice being able to login from anywhere and write a new post or continue working on a draft. You can’t possibly do that with a static site generator? Wrong! With my current site, I’m able to use Github’s UI to create new files or make changes to existing files. If you’re using Github Pages or a deploy system like my site, any changes that are make will automatically get built and deployed. I’ve even seen iPhone and Android apps that allow you to do the same thing assuming your site code is in a Github repo.

Making The Switch

I hope I’ve at least interested you in static site generators. In the next post I’m going to walk through setting up a Jekyll site and Github Pages.

Supporting Multiple Versions of Symfony Components

Symfony recently released version 3.0 of their components and framework. I use these components in a couple of my own packages, along with a bunch of other packages. How do we allow our package to work with multiple versions of the Symfony components?

If your code is compatible in both 2.x and 3.0, you can support both of those in your composer.json file. If not, you will likely want to manage different releases of your library and let the user know which one to use for their version of Symfony.

We can use a pipe operator as an or in composer. For example in my FileLoader package, I use the Config and YAML components and I want to support 2.3 and up. In my composer.json file, I changed these lines.

// Old
"symfony/config": "~2.3",
"symfony/yaml": "~2.3"

// New
"symfony/config": "~2.3|~3.0",
"symfony/yaml": "~2.3|~3.0"

If the project already has an existing version of the Symfony component, it will use that, otherwise it will use the latest allowed, which in this case is 3.0.

Testing

Our package now supports multiple versions, but how do we verify that it works in all those packages? We’re going to look at how to do this in TravisCI. If you’re not using Travis, I would highly recommend it for open source projects. It makes continuous integration a breeze.

Let’s take a look at a basic .travis.yml file for a PHP project.

language: php

php:
  - '5.4'
  - '5.5'
  - '5.6'
  - '7.0'
  - hhvm

sudo: false

before_script:
  - composer install --dev --no-interaction

With this it will only test against the latest version of Symfony components in each PHP version we specify. We can use environment variables to test against multiple versions of Symfony components. Let’s add one for each version we are allowing in our composer.json, in this case 2.3 and above.

env:
  - SYMFONY_VERSION=2.3.*
  - SYMFONY_VERSION=2.4.*
  - SYMFONY_VERSION=2.5.*
  - SYMFONY_VERSION=2.6.*
  - SYMFONY_VERSION=2.7.*
  - SYMFONY_VERSION=2.8.*
  - SYMFONY_VERSION=3.0.*

Then in our before_script section, we’re going to tell composer to require that version of Symfony before we run composer.

before_script:
  - composer self-update
  - if [ "$SYMFONY_VERSION" != "" ]; then composer require "symfony/symfony:${SYMFONY_VERSION}" --no-update; fi;
  - composer update

We also told composer to self-update, as travis is running an old version of composer.

For each one of the env values and PHP version we are testing against, Travis will create a build. This is our build Matrix. Every time we push, it will create our Matrix and look something like this.

Travis builds

Our Matrix can also be customized manually using the matrix option in our Travis config. We are going to do that to tell Travis not to create a build for PHP 5.4 and Symfony 3.0, as Symfony 3.0 requires PHP 5.5 or above.

matrix:
  exclude:
    - php: '5.4'
      env: SYMFONY_VERSION=3.0.*

You can add any number of things to exclude or even include here. Just check out the travis docs.

One final thing we are going to do is cache the global Composer directory. This will allow Composer to use a cached version of the Symfony component, rather than having to fetch it every time.

With all that said and done, our completed .travis.yml file should look like this.

language: php

php:
  - '5.4'
  - '5.5'
  - '5.6'
  - '7.0'
  - hhvm

sudo: false

cache:
  directories:
    - $HOME/.composer

env:
  - SYMFONY_VERSION=2.3.*
  - SYMFONY_VERSION=2.4.*
  - SYMFONY_VERSION=2.5.*
  - SYMFONY_VERSION=2.6.*
  - SYMFONY_VERSION=2.7.*
  - SYMFONY_VERSION=2.8.*
  - SYMFONY_VERSION=3.0.*

matrix:
  exclude:
    - php: '5.4'
      env: SYMFONY_VERSION=3.0.*

before_script:
  - composer self-update
  - if [ "$SYMFONY_VERSION" != "" ]; then composer require "symfony/symfony:${SYMFONY_VERSION}" --no-update; fi;
  - composer update

Commit, push, and verify that your code works with all versions of the Symfony components.

Bonus: Getting Coveralls Working

If you are using php-coveralls and happen to be using the Config Symfony component (like FileLoader is), then you will not be able to test against Symfony 2.3, as php-coveralls requires 2.4 or later. To fix this, I’ve opted to install it globally in composer. You could also download the .phar and use that instead.

after_success:
  - composer global require satooshi/php-coveralls
  - travis_retry php $HOME/.composer/vendor/bin/coveralls -v

Update 12/27/15: As some commenters have pointed out, Symfony 2.4, 2.5, and 2.6 are no longer supported. You can skip these in your Travis builds if you please.

Boxen Tips and Tricks

Automate all the things!

A while back I wrote about The Perfect Dev Setup that walked through what I considered to be (at the time), the perfect development setup. Part of the reason I wrote a post about it so that I could repeat it on any machine that I used, but it would still take time and copy/pasting commands. Surely there must be a better way?

Spoiler alert, I started automating this with Boxen. Boxen it a tool that Github released a couple years back that automated new employee’s computers. It was designed for Mac, which is the platform that I use. It uses Puppet, which is a provisioning tool. That means if you know Puppet, you can really do some cool stuff with it, and if not, I think it’s easy enough to understand to use.

Getting Boxen setup is really easy. Just follow the Readme and you’re good to go. There are tons of modules that give you almost anything that you would need.

Here are some tips and tricks I’ve learned using Boxen.

Disable Full Disk Encryption

By default Boxen requires you to have Full Disk Encryption enabled. You can get by that requirement by using the --no-fde option when running boxen.

$ boxen --no-fde

Multiple Computers

If you’re anything like me, you probably have multiple computers. You’re main computer, a laptop, and probably a work computer. Being able to setup multiple machines and keep changes across those is what Boxen great. But what about when you want something on your work machine, but not on your laptop? Or what about something you only want on your personal machines? Boxen ships with the people module, but that assumes different usernames.

Boxen is built on Puppet, so we can use Node Definitions. By default manifests/site.pp only has a default node, but we can add as many as we want. Let’s create a node for each of our computers.

node 'Matthews-Mac.local' {
    # Only runs on iMac
}

node 'Matthews-Laptop.local' {
    # Only runs on laptop
}

node 'Matthew-OSX.local' {
    # Only runs on work computer
}

node default {
}

Now we can copy everything from default and add it to each node and customize it.

Note

I recommend keeping the default node around. If you don’t have it and no other nodes match, Puppet will fail.

Using Modules to Abstract Common Things

Having multiple nodes is awesome. Now we can customize what we need for each computer, but you might find that you’re using some of the same things over and over. Having to manage those things in multiple places is work. Let’s fix this by moving those to modules.

If you don’t know Puppet, don’t worry, I’ll walk you through a basic module structure. If you’re feeling ambitious, check out Puppet’s Module Guide.

I’m going to call my module common following the best practice. The structure for a module will look something like this:

/modules
  /common
    /files
      php.ini
    /manifests
      /init.pp
      /development.pp
    /templates
      nginx.conf.erb

At minimum you just need the manifests/init.pp file.

Let’s start off by moving everything to our common module.

# modules/common/manifests/init.pp
class common {
  # core modules, needed for most things
  include dnsmasq
  include git
  include hub
  include nginx

  # fail if FDE is not enabled
  if $::root_encrypted == 'no' {
    fail('Please enable full disk encryption and try again')
  }

  # node versions
  nodejs::version { 'v0.6': }
  nodejs::version { 'v0.8': }
  nodejs::version { 'v0.10': }

  # default ruby versions
  ruby::version { '1.9.3': }
  ruby::version { '2.0.0': }
  ruby::version { '2.1.0': }
  ruby::version { '2.1.1': }
  ruby::version { '2.1.2': }

  # common, useful packages
  package {
    [
      'ack',
      'findutils',
      'gnu-tar'
    ]:
  }

  file { "${boxen::config::srcdir}/our-boxen":
    ensure => link,
    target => $boxen::config::repodir
  }
}

Then update our nodes to include this.

node 'Matthews-Mac.local' {
    # Only runs on iMac
    include common
}

node 'Matthews-Laptop.local' {
    # Only runs on laptop
   include common
}

node 'Matthew-OSX.local' {
    # Only runs on work computer
    include common
}

node default {
    include common
}

Let’s move our development stuff to common::development.

# modules/common/manifests/development.pp
class common::development {
  include mysql
  include redis
  include virtualbox
  include vagrant

  # node versions
  nodejs::version { 'v0.8': }
  nodejs::version { 'v0.10': }
  nodejs::version { 'v0.12.0': }

  class { 'nodejs::global': version => 'v0.12.0' }

  # default ruby versions
  ruby::version { '1.9.3': }
  ruby::version { '2.0.0': }
  ruby::version { '2.1.0': }
  ruby::version { '2.1.1': }
  ruby::version { '2.1.2': }
}

Then update our nodes.

node 'Matthews-Mac.local' {
    # Only runs on iMac
    include common
    include common::development
}

node 'Matthews-Laptop.local' {
    # Only runs on laptop
   include common
}

node 'Matthew-OSX.local' {
    # Only runs on work computer
    include common
    include common::development
}

node default {
    include common
}

You can do this with any number of things. A module can have any number of classes. That means if you wanted to separate even more things, you could easily do that.

In additional to modules, you can also continue to make changes individually to nodes. That means if you need Node.js on your laptop, but only want v0.10.26, you can just include that in the node without having all your development items.

Bonus: Custom PHP Project Module

As a bonus, I’ve moved out my PHP specific items from my common module and included some usages. You can see them at this Gist.

Hiera

Sometimes you need a module to install a different version of a package, or install in a different location. Instead of forking and making that change manually in a module, some modules support Hiera for changing these. You can read more about it in the Boxen Readme.

For example, I can’t use the .dev tld that dnsmasq uses by default at work, so instead I use .localhost. To change that, it’s one line I have to add.

# hiera/common.yaml
---
dnsmasq::tld: localhost

No need to fork and make that change in the code.

Hacking Further

Once you get the hang of Puppet, you can start to do some really cool things in Boxen. I wrote a module that installs pyenv to let you manage different versions of Python.

Let me know in the comments if you have any tips or tricks with Boxen, or any cool Boxen modules you’ve written.