Testing an ASP.NET MVC app with Ruby

Why testing through the UI is hard(ly done)

Writing UI tests for a web application can be quite cumbersome and is a painstakingly repetitive task. Also, the UI of an application tends to change often during development, making UI tests a brittle resource. To top this, when running UI tests, you need to count on a browser to act as a host for them, making them a lot slower than regular unit tests.

It is for these and other reasons that people often don't go through the trouble of writing UI tests. And they are, in my opinion, right to do so.

But still, I have seen a lot of applications break in the UI after making minor changes. If these errors don't pop up in your staging environment, you are left with a red face once you deploy to production (and yup, I have seen this happening too often, to not take steps to fix this). And no, people don't always go through the entire application when it's on the staging environment. The times where I've seen full test plan execution for an application, before it is allowed to go through to production, can be counted on (less than) one finger. The only fallback we have here are our unit tests and as the name says, they are for testing (small) units, not entire applications (read: end-to-end testing).

There are of course testing frameworks you can use to run your UI tests, Selenium being one of them. I have been using Selenium on a couple of projects, but often give up quickly because of the slowness of the tests, because tests are hard to maintain, ... and so on.

Now, I recently followed the Ruby and Rails for n00bs session of my good friend Michel (@michelgrootjans) and got introduced to a couple of Ruby test gems that actually open up a couple of opportunities you can use in your .NET applications as well.

In this blog post I want to give you an overview of how you can set these Ruby test gems up, so they can run UI tests for an ASP .NET MVC application (or any .NET web application that is). The technologies (and gems) I will be describing here are:

• Ruby (of course)
• RSpec
• Capybara
• Selenium
• PhantomJS
• Poltergeist 

Setting up the test environment

First of all, let's get our environment setup. For this, you will need Ruby, which you can find here. Once installed, you can test if all went well by running the following command from the command line:

irb

This should open up a Ruby REPL loop where you can test out statements (for instance 1+1, should give you the answer of 2).

Once this is setup. Exit the REPL loop and start installing all the necessary gems:

gem install rspec
gem install capybara

gem install selenium-webdriver
gem install poltergeist

One other thing we now need to install is PhantomJS, which you can find here and Firefox (which is used by the default selenium web driver, I will give you other options further down this blog post). And that's all we need to get started.

Writing a first test with rspec

We can now start writing tests. Our tests will be placed in separate _spec.rb files in a spec folder. For instance if you want to write specification tests for customers, you will have a customers_spec.rb file, for products, you will have a products_spec.rb file. Of course you are free to use whatever naming and grouping of your specification tests, just make sure you have a spec folder with in it at least one _spec file.

A spec looks like this:

describe 'adding a new customer' do
  it 'should be able to create a new customer' do
  end
end

As you can see, you start of with a describe, which tells what kind of behavior you want to test. The it part contains the expected outcome (we will add this later). Don't worry about the Ruby syntax for now. For people using a BDD-style testing framework the describe and it syntax should look familiar.

The things we will want to test, are going to be the behavior of a website. For this we will use the capybara and selenium gems. And since we want this first spec and all following specs to be using the same setup for our tests (eg. url of the site we will be testing), we will use a spec_helper.rb file for this. In this file we will require all the necessary gems and do all of the setup. Each individual spec file will then require this one spec_helper file:

require 'capybara/rspec'
require 'selenium/webdriver'

Capybara.run_server = false
Capybara.default_driver = :selenium
Capybara.app_host = 'http://localhost/TheApplication'

After the require statements, we configure capybara. First of all, we tell it to not run its server. Since capybara is essentially a rails testing framework, it runs under a rack application by default. We won't have this, since our app will be running in IIS (Express) or something similar. Next we tell it which driver to use and last where our application is located (you could test www.google.com if you'd like).

Now that we have this setup, we can start writing a first test. Capybara actually allows you to click buttons and fill out fields in a web application, and that's what we will be doing:

require 'spec_helper'

describe 'adding a new customer', :type => :feature do
  it 'should be able to create a new customer' do
    visit '/Customer'

    click_link 'Add'

    fill_in('first_name', :with => 'Gitte')
    fill_in('last_name', :with => 'Vermeiren')
    fill_in('email', :with => 'mie@hier.be')

    click_button 'Create'

    page.should have_text 'New customer created'
  end
end

As you can see, capybara has methods for clicking links, filling out fields, choosing options, ... and in the end for testing whether you get an expected outcome in your application. You can find the capybara documentation with additional capabilities here.

Also notice I added the :type => :feature hash to the describe, so the test will be picked up as an rspec test.

Once you have this spec file, you can actually run your test. For this fire up a command prompt, cd to the directory just above your spec directory and run the rspec command:

rspec

You will notice this command will take a bit to start up, but once it's running, it will start up a firefox window and show you an error in the command prompt, since I assume you don't have anything implemented yet to get the above test to succeed. I leave it up to you, the reader, to write a small web app to get this test passing. Once you have this and you run the rspec command, you will see the different fields in your browser filled out with the values you specified in your test.

Improving our test

Now, I promised you, the return on investment with rspec, capybara, ruby, ... would be worth the effort. First of all, I can tell you from experience that the above capybara test is a lot cleaner than comparable tests written in .NET. But above that, we can do more. 

We can start of by having our tests run headless, meaning that we won't be needing a browser window anymore. For this we will use poltergeist and phantomjs. You will need to alter the spec_helper file for this:

require 'capybara/rspec'
require 'selenium/webdriver'
require 'capybara/poltergeist'

Capybara.run_server = false
Capybara.default_driver = :selenium
Capybara.javascript_driver = :poltergeist
Capybara.app_host = 'http://localhost/TheApplication'

Capybara.register_driver :poltergeist do |app|
  Capybara::Poltergeist::Driver.new(app,
                                    :phantomjs => 'C:\\some_path\\phantomjs.exe')
end

This adds the extra requirement for poltergeist and adds the javascript_driver configuration setting. Additionally we tell capybara where it can find the phantomjs process.

You will also need to add the :js => true hash to your describe statement for it to run in phantomjs:

describe 'adding a new customer', :type => :feature, :js => true  do

If you run your tests now with rspec, you will notice, firefox will not run, you will just get a summary in the command prompt of which tests succeeded or failed.

I ran some tests with our own web application and noticed that the headless tests ran twice as fast as the tests using the browser.

Further improvements

I am actually already quite happy with this first test setup. It's still not the fastest running tests, but it does allow me to get in some easy end-to-end testing, or to even start doing some BDD style testing with this. You can also use cucumber if you'd want to to instead off rspec.

In my own setup I extended the above example with some extra configuration settings to be able to easily switch my testing environment from my local to the staging environment and from headless to browser testing.

I also did some tests with chrome (doable) and IE (very, very slow test runs!), but for now prefer the firefox and headless setup.

I would also like to add some extra stuff to setup and clear my database before and after each test. That is something I haven't figured out yet, but should be easily added.