Capybara and Selenium with Vagrant

Capybara is an easy way to perform integration testing through a browser for your web applications. Capybara has various drivers including one which works with Selenium Webdriver that allows you to run your tests against a number of different browsers including Internet Explorer, Firefox and Chrome. When you develop directly on your own machine then all is good and running your tests through Capybara with Selenium will fire up a browser window to run your tests. I like to develop web applications using Vagrant boxes with Ubuntu server on them and there the same approach won’t work directly, in this article I’ll show what you can do to automate testing using Capybara with Selenium on such an environment.

For the examples in this article I’ll use the famous Echo example in Sinatra form: a web application on which you post a message which it then simply echoes back at you. The full source code of is available on GitHub at https://github.com/mkremer/echo, to get it up and running simply clone the repository and run “vagrant up” (assuming you have Vagrant installed already) inside it to setup and bring up the Vagrant box (note that this uses Vagrant’s precise64 base box, and it will download that box if you don’t have it already which can take a little while depending on your internet connection). To follow along with the article you’ll only need to edit test/test_helper.rb and test/echo_test.rb using snippets in the article, the git repository contains the final version of those files with verbose comments added.

We’ll work with a single test case (defined in test/echo_test.rb) that posts a message and asserts that the message is echoed back:

require "test_helper"

class EchoTest < CapybaraTestCase
  def test_echo
    visit "/"
    fill_in "message", with: "Hello, World!"
    click_on "Submit"
    assert page.has_content? "Echo: Hello, World!"
  end
end

The test suite can be executed on by running “bundle exec rake test”.

Run Selenium and browsers in your Vagrant box

To run a browser on your Vagrant box you’ll need to run an X server. On servers you usually don’t want to run a desktop environment, instead you can use Xvfb which is a “fake” X server which is often used for testing. Driving Xvfb through Ruby can be done easily using the headless gem.

The Vagrant box of the Echo project already has Xvfb and browsers installed through its provisioning script, so we can get straight to the code. Below are the contents to put in test/test_helper.rb to enable the ability to run using Xvfb:

require_relative "../echo"
require "capybara/dsl"
require "headless"
require "minitest/autorun"

# CAPYBARA_DRIVER is the Capybara driver to use, this defaults to Selenium with
# Firefox
ENV["CAPYBARA_DRIVER"] ||= "selenium_firefox"

# Setup Capybara to work with the Echo application
Capybara.app = Echo

# Set the default driver to CAPYBARA_DRIVER, when you change the driver in one
# of your tests resetting it to the default will ensure that its reset to what
# you specified when starting the tests.
Capybara.default_driver = ENV["CAPYBARA_DRIVER"].to_sym

# CapybaraDriverRegistrar is a helper class that enables you to easily register
# Capybara drivers
class CapybaraDriverRegistrar
  # register a Selenium driver for the given browser to run on the localhost
  def self.register_selenium_local_driver(browser)
    Capybara.register_driver "selenium_#{browser}".to_sym do |app|
      Capybara::Selenium::Driver.new(app, browser: browser)
    end
  end
end

# Register various Selenium drivers
CapybaraDriverRegistrar.register_selenium_local_driver(:firefox)
CapybaraDriverRegistrar.register_selenium_local_driver(:chrome)

# Base test case class to use for tests using Capybara
class CapybaraTestCase < MiniTest::Unit::TestCase
  include Capybara::DSL

  # setup is run before each test and sets up the environment appropriately,
  # the code should pretty much explain itself
  def setup
    if headless?
      @headless = Headless.new
      @headless.start
    end
  end

  # teardown is run after each test, it saves a screenshot if the last test
  # failed and resets Capybara for the next test
  def teardown
    Capybara.reset_sessions!
    Capybara.use_default_driver
  end

  # Determines if the test runs on a headless environment
  def headless?
    ENV["GUI"] == "headless"
  end
end

The above code assumes that there are developers who run the project directly on their own machine as well as those who run it using Vagrant, and it will only use Xvfb if the environmental variable GUI is set to headless. If you take a look at the provisioning script you’ll notice that this variable is automatically set to headless in the vagrant user’s environment (assuming the user is either using zsh or bash) so you won’t have to set it manually when running your tests on the Vagrant box. When we now run the tests on the Vagrant box with “bundle exec rake test” Firefox will be run in Xvfb and you’ll see nothing other than an uninspiring dot that indicates the test has passed. The code includes the option to run with Chrome (in the form of Chromium) instead of Firefox, this can be done by setting the environmental variable CAPYBARA_DRIVER to selenium_chrome.

If you want to see what happens during your test you can have Selenium take screenshots. Here’s the CapybaraTestCase class changed to save a screenshot in test/tmp when a test fails:

# Base test case class to use for tests using Capybara
class CapybaraTestCase < MiniTest::Unit::TestCase
  include Capybara::DSL

  # setup is run before each test and sets up the environment appropriately,
  # the code should pretty much explain itself
  def setup
    if headless?
      @headless = Headless.new
      @headless.start
    end
  end

  # teardown is run after each test, it saves a screenshot if the last test
  # failed and resets Capybara for the next test
  def teardown
    save_screenshot(__name__) unless passed?
    Capybara.reset_sessions!
    Capybara.use_default_driver
  end

  # Determines if the test runs on a headless environment
  def headless?
    ENV["GUI"] == "headless"
  end

  # Saves a screenshot in PNG format with a timestamp and the given name in test/tmp/
  # Note that this code assumes that you're using a Selenium driver
  def save_screenshot(name)
    filename = File.join(File.dirname(__FILE__), "tmp", "#{Time.now.strftime("%Y-%m-%d-%H-%M-%S")}_#{name}.png")
    page.driver.browser.save_screenshot(filename)
  end
end

If you now break the test case by inverting the assertion you’ll find that a screenshot is saved in test/tmp to help you figure out what went wrong.

Run Selenium and browsers outside the Vagrant box

Using an X server on your Vagrant box limits you to running your tests against browsers that you can run on it, another option is to run Selenium Server outside your Vagrant box directly on your machine and have the Selenium driver connect to that instance instead. This gives you the same advantages as if you were running the entire code base locally, it enables you to watch the tests as they run (which can be useful to figure out problems that aren’t apparent from screenshots) and if you add a breakpoint somewhere in your test you can inspect your web pages directly in your own browser.

Below is the test_helper changed to allow you to connect to other Selenium Server instances:

require_relative "../echo"
require "capybara/dsl"
require "headless"
require "minitest/autorun"

# SELENIUM_SERVER is the IP address or hostname of the system running Selenium
# Server, this is used to determine where to connect to when using one of the
# selenium_remote_* drivers
ENV["SELENIUM_SERVER"] ||= "192.168.33.1"

# SELENIUM_APP_HOST is the IP address or hostname of this system (where the
# tests run against) as reachable for the SELENIUM_SERVER. This is used to set
# the Capybara.app_host when using one of the selenium_remote_* drivers
ENV["SELENIUM_APP_HOST"] ||= "192.168.33.10"

# CAPYBARA_DRIVER is the Capybara driver to use, this defaults to Selenium with
# Firefox
ENV["CAPYBARA_DRIVER"] ||= "selenium_firefox"

# Setup Capybara to work with the Echo application
Capybara.app = Echo

# Set the default driver to CAPYBARA_DRIVER, when you change the driver in one
# of your tests resetting it to the default will ensure that its reset to what
# you specified when starting the tests.
Capybara.default_driver = ENV["CAPYBARA_DRIVER"].to_sym

# CapybaraDriverRegistrar is a helper class that enables you to easily register
# Capybara drivers
class CapybaraDriverRegistrar
  # register a Selenium driver for the given browser to run on the localhost
  def self.register_selenium_local_driver(browser)
    Capybara.register_driver "selenium_#{browser}".to_sym do |app|
      Capybara::Selenium::Driver.new(app, browser: browser)
    end
  end

  # register a Selenium driver for the given browser to run with a Selenium
  # Server on another host
  def self.register_selenium_remote_driver(browser)
    Capybara.register_driver "selenium_remote_#{browser}".to_sym do |app|
      Capybara::Selenium::Driver.new(app, browser: :remote, url: "http://#{ENV["SELENIUM_SERVER"]}:4444/wd/hub", desired_capabilities: browser)
    end
  end
end

# Register various Selenium drivers
CapybaraDriverRegistrar.register_selenium_local_driver(:firefox)
CapybaraDriverRegistrar.register_selenium_local_driver(:chrome)
CapybaraDriverRegistrar.register_selenium_remote_driver(:firefox)
CapybaraDriverRegistrar.register_selenium_remote_driver(:chrome)
CapybaraDriverRegistrar.register_selenium_remote_driver(:internet_explorer)

# Base test case class to use for tests using Capybara
class CapybaraTestCase < MiniTest::Unit::TestCase
  include Capybara::DSL

  # setup is run before each test and sets up the environment appropriately,
  # the code should pretty much explain itself
  def setup
    if headless?
      @headless = Headless.new
      @headless.start
    end

    if selenium_remote?
      Capybara.app_host = "http://#{ENV["SELENIUM_APP_HOST"]}:#{page.driver.rack_server.port}"
    end
  end

  # teardown is run after each test, it saves a screenshot if the last test
  # failed and resets Capybara for the next test
  def teardown
    save_screenshot(__name__) unless passed?
    Capybara.reset_sessions!
    Capybara.use_default_driver
    Capybara.app_host = nil
  end

  # Determines if a selenium_remote_* driver is being used
  def selenium_remote?
    Capybara.current_driver.to_s =~ /\Aselenium_remote/
  end

  # Determines if the test runs on a headless environment
  def headless?
    ENV["GUI"] == "headless" && !selenium_remote?
  end

  # Saves a screenshot in PNG format with a timestamp and the given name in test/tmp/
  # Note that this code assumes that you're using a Selenium driver
  def save_screenshot(name)
    filename = File.join(File.dirname(__FILE__), "tmp", "#{Time.now.strftime("%Y-%m-%d-%H-%M-%S")}_#{name}.png")
    page.driver.browser.save_screenshot(filename)
  end
end

The CapybaraDriverRegistrar has been modified to enable you to register remote drivers with the new register_selenium_remote_driver method. When using such a remote driver (set through the CAPYBARA_DRIVER environmental variable) the above code connects to a Selenium Server running on 192.168.33.1 which in the Echo project’s setup is your own machine. Additionally it tells Capybara to build its URLs with 192.168.33.10 (the IP address of the Vagrant box in the Echo project) instead of 127.0.0.1 (which is the default). The IP address (or hostname) of the Selenium Server and that of the application host used by Capybara to write its URLs can be overridden using the environmental variables SELENIUM_SERVER and SELENIUM_APP_HOST. Note that for this setup the Vagrant box uses host-only networking, and thus the Vagrant box only exists on the network for your own machine and virtual machines you run inside it. If you want other machines to be able to access your Vagrant box you can use bridged networking.

Capybara itself includes Selenium Server however that’s installed somewhere in the Vagrant box, to get Selenium running on your machine (or elsewhere) you will have to download Selenium Server and run it with “java -jar selenium-server-standalone-2.24.1.jar” (there is a version number filename, when you download it you may get a newer version and thus a slightly different filename). Once you’re running Selenium Server locally run “CAPYBARA_DRIVER=selenium_remote_firefox bundle exec rake test” on the Vagrant box and you should see the test case run in a Firefox window on your local machine. Certain browsers (such as Chrome and Opera) will require you to install additional drivers so be sure to checkout the Selenium website for more information.

Wrapping up

Now that we’ve come to the end of this article the code is capable of running with Selenium in different ways by setting environmental variables, different developers can run the integration tests according to their own preferences and your CI environment can also reap the benefits. Hopefully this example will be of use for your projects.

Advertisements
Advertisements
%d bloggers like this: