software solutions / project leadership / agile coaching and training

Using Cucumber for unit tests… why not?

Posted on December 13, 2010

It seems that the accepted way to test in Ruby is to use Rspec for unit tests and to use Cucumber for acceptance tests (higher level functional testing). After doing a little bit of Cucumber, I’ve started to fall in love with the format of Cucumber tests.

Most Rubyists would probably agree that behavior-driven development is good (in other words, writing tests in a Given/When/Then format). We obviously do this in Cucumber (there isn’t much choice), but I’ve also written tests in this format in Rspec and in .NET.

I like BDD for two main reasons. First, I believe that software development is a series of translations. I want to translate business requirements into readable, executable specifications, then translate that into tests, then translate that into implementation code. Second, before I implement a feature and even before I write my tests, I try to write out what I want the code to do in English. If I can’t write out what I want to do in English, how and I supposed to know what I’m supposed to write in code?

Here’s my theory: if we agree that BDD is good, why don’t we write our unit tests in a format that is more amenable to BDD, that being the Cucumber format of tests? I’m not saying that we write acceptance level tests instead of unit tests, I’m saying that maybe we should write unit tests in a different format. Not only that, Cucumber tables give us a nice way to write more readable, data-driven tests. Here are a couple examples from the supermarket pricing kata (in Rspec and Cucumber).

Cucumber:

Feature: Checkout

Scenario Outline: Checking out individual items
Given that I have not checked anything out
When I check out item
Then the total price should be the of that item

Examples:
| item | unit price |
| "A" | 50 |
| "B" | 30 |
| "C" | 20 |
| "D" | 15 |

Scenario Outline: Checking out multiple items
Given that I have not checked anything out
When I check out
Then the total price should be the of those items

Examples:
| multiple items | expected total price | notes |
| "AAA" | 130 | 3 for 130 |
| "BB" | 45 | 2 for 45 |
| "CCC" | 60 | |
| "DDD" | 45 | |
| "BBB" | 75 | (2 for 45) + 30 |
| "BABBAA" | 205 | order doesn't matter |
| "" | 0 | |

Scenario Outline: Rounding money
When rounding "" to the nearest penny
Then it should round it using midpoint rounding to ""

Examples:
| amount | rounded amount |
| 1 | 1 |
| 1.225 | 1.23 |
| 1.2251 | 1.23 |
| 1.2249 | 1.22 |
| 1.22 | 1.22 |

Rspec:

require 'spec_helper'

describe "Given that I have not checked anything out" do
before :each do
@check_out = CheckOut.new
end

[["A", 50], ["B", 30], ["C", 20], ["D", 15]].each do |item, unit_price|
describe "When I check out an invididual item" do
it "The total price should be the unit price of that item" do
@check_out.scan(item)
@check_out.total.should == unit_price
end
end
end

[["AAA", 130], # 3 for 130
["BB", 45], # 2 for 45
["CCC", 60],
["DDD", 45],
["BBB", 75], # (2 for 45) + 30
["BABBAA", 205], # order doesn't matter
["", 0]].each do |items, expected_total_price|
describe "When I check out multiple items" do
it "The total price should be the expected total price of those items" do
individual_items = items.split(//)
individual_items.each { |item| @check_out.scan(item) }
@check_out.total.should == expected_total_price
end
end
end
end

class RoundingTester
include Rounding
end

[[1, 1],
[1.225, 1.23],
[1.2251, 1.23],
[1.2249, 1.22],
[1.22, 1.22]].each do |amount, rounded_amount|
describe "When rounding an amount of money to the nearest penny" do
it "Should round the amount using midpoint rounding" do
RoundingTester.new.round_money(amount).should == rounded_amount
end
end
end

A couple things stand out to me when you compare these two. First, if I want to run data-driven tests with different values, the Cucumber syntax is so much cleaner and more descriptive. Second, the “Given I have not checked anything out” section in the Rspec version is really long and contains two nested “describe” sections (many times you end up with many more than this). When you nest sections like this, it’s really hard to see the context of things or read the tests because the “Given” text is nowhere near the nested “When” sections in the code.

Rspec follows in the footsteps of previous unit testing frameworks that write test methods in test classes (or in the case of Rspec, something that resembles test classes and methods. But is this the best way, or just the way that we’re used to? We have been writing unit tests this way for years and years because we had no other choice. But that doesn’t mean that it’s the best way.

Here are the benefits I see of using the Cucumber syntax over Rspec:

  • The tests are much easier to read (especially when doing data-driven “scenario outline” tests).
  • The Given/When/Then text is all in one place (as opposed to spread out and mixed in with code).
  • It forces me to be able to write out in English what I want the code to do.
  • Any step definition that I write can easily be reused anywhere in any other Cucumber test.
  • The code just looks cleaner. I’ve seen a lot of messy Rspec tests.
  • Rspec doesn’t have a method that corresponds to the “When” step (unless I’m missing something), so you have to shoehorn it into before(:each) or the “it” method. (I’m not sure why this is, we figured this out in the .NET world long ago.)

To be fair, there are more BDD-friendly flavors of Rspec (like rspec-given). This helps you write tests in Given/When/Then format, but I still feel like all of the underscores and symbols and syntax is getting in the way of the actual test verbiage.

Favoring Cucumber is my personal preference and I know that there are some people that would probably disagree with my opinion on this, and that’s fine. But I’m really enjoying what Cucumber brings to the table, both in terms of functionality and the syntax.

10 Comments »

  1. RSpec examples can be quite clean if you understand the tools at your disposal. Here’s how I’d do the checkout examples:

    https://gist.github.com/741672

    It’s a translation of the test/unit tests on http://codekata.pragprog.com/2007/01/kata_nine_back_.html, but has the added benefit that each of the “calculating totals” examples runs separately, so you get more granular feedback as you go through the kata.

    It’s admittedly not GWT, but GWT is not a requirement of BDD in my view: it’s just a mechanism to nudge you in the right direction.

    David Chelimsky — December 15, 2010 @ 12:54 am

  2. Hi Jon,

    I definitely agree with you with that. Even RSpec is cleanly written like David has done, cucumber still shines.

    Have you done more unit testing through cucumber by the way in your projects etc.?

    Regards

    Inanc Gumus — January 26, 2011 @ 5:28 pm

  3. Inanc,

    I’m not currently doing unit tests in Cucumber on my current project just because we’ve been using RSpec and I want to keep things consistent. But if I were doing a new project, I would have more Cucumber unit tests.

    Jon Kruger — January 26, 2011 @ 7:04 pm

  4. Hi John- I am looking for a place to get Cucumber certification. Do any of the training companies in Columbus offer it?

    Thanks- Ray

    Ray Zink — January 17, 2012 @ 10:52 am

  5. I have been reading about Cucumber for the past two days. Every web-article I have read describes creating applications (mostly Ruby on Rails) from the ground up using Cucumber. I have yet to find a single article that even hints at using Cucumber to test any aspect of an existing RoR application.

    I’m interested to know if you are aware of anyone who has taken a completed RoR application and then started to test it, in an ongoing manner, using Cucumber. From what I understand, it would be a very challenging process to create scenarios to match the existing application functions. (Then there would be the added challenge of going to the end user and getting their input.)

    If you are aware of any articles about using Cucumber to test an application that is already complete, but was developed without Cucumber, I would appreciate receiving links to them.

    Thank you,
    ~ Allen

    Allen Roulston — October 10, 2012 @ 4:54 pm

  6. @Allen,

    Actually Cucumber works great for existing applications because you’re testing from the outside by driving a web browser, calling a web service, etc. It doesn’t matter if the code being tested is horrible because you’re just testing through the interface. The approach to testing is pretty much the same for testing new applications as it is for existing applications.

    Jon Kruger — October 10, 2012 @ 8:25 pm

  7. Hello Jon,

    would be able to “hook me up” with a good tutorial that shows Cucumber driving a web browser? In the past two days of web surfing research I have not encountered a single reference to such testing with Cucumber. The only articles I have found are discussing application development from ground zero with Cucumber.

    I appreciate your response,
    ~ Allen

    Allen Roulston — October 11, 2012 @ 9:39 am

  8. Allen,

    Testing legacy code:
    https://www.google.com/search?q=cucumber+test+legacy&rlz=1C1CHFA_enUS484US484&oq=cucumber+test+legacy&sugexp=chrome,mod=10&sourceid=chrome&ie=UTF-8

    Selenium and Watir are two tools you can use to drive a browser.

    Here’s a good book to help you out:
    https://leanpub.com/cucumber_and_cheese

    Jon Kruger — October 11, 2012 @ 9:10 pm

  9. Hello Jon,

    Your suggestions are appreciated.
    I am now back on the research trail.

    ~ Allen

    Allen Roulston — October 15, 2012 @ 10:24 am

  10. Hi Jon,

    And, after 2 years any new thoughts or experience on this topic? Still a relevant question/observation and after having done some Cucumber work I also think that cucumber might be very help full in specifying at unit test level.

    An update would be very much appreciated.

    Kind regards,

    Willem

    Willem Dijkgraaf — March 21, 2014 @ 9:16 am

Leave a comment





SERVICES
SOFTWARE SOLUTIONS
I have over 10 years of software development experience on several different platforms (mostly Ruby and .NET). I recognize that software is expensive, so I'm always trying to find ways to speed up the software development process, but at the same time remembering that high quality is essential to building software that stands the test of time.
PROJECT LEADERSHIP
I have experience leading and architecting large Agile software projects and coordinating all aspects of a project's lifecycle. Whether you're looking for technical expertise or someone to lead all aspects of an Agile project, I have proven experience from multiple projects in different environments that can help make your project a success.
AGILE COACHING
I believe that Agile processes and tools should be applied with common sense. I've spent the last 6 years working on Agile projects as a consultant in many different environments, both in leadership roles and as a practitioner doing the work. I can help you find out how Agile can work best in your organization, not just apply a prescriptive process.
TEST DRIVEN DEVELOPMENT TRAINING
TDD Boot Camp is a hands-on, three day, comprehensive training course that will teach you all of the skills, tools, frameworks that you will need to use test-driven development to develop real world .NET applications. If you're not looking for something that intensive, check out the the half-day version.
Have any questions? Contact me for more information.
PRESENTATIONS
The Business of You: 10 Steps For Running Your Career Like a Business
From CONDG 2012, Stir Trek 2014
From Stir Trek 2013, DogFoodCon 2013
From Stir Trek 2012, QA or the Highway 2014
(presented with Brandon Childers, Chris Hoover, Laurel Odronic, and Lan Bloch from IGS Energy) from Path to Agility 2012
(presented with Paul Bahler and Kevin Chivington from IGS Energy)
From CodeMash 2011
An idea of how to make JavaScript testable, presented at Stir Trek 2011. The world of JavaScript frameworks has changed greatly since then, but I still agree with the concepts.
A description of how test-driven development works along with some hands-on examples.
From CodeMash 2010
From CodeMash 2010