Changing how I structure unit tests

Posted on January 18th, 2012 in .NET, TDD, unit testing by Jon Kruger

When I write unit tests, I use the BDD-style GIven/When/Then format so that the tests are descriptive and explain the business functionality that is being implemented. But I’ve recently changed the way that I do it.

Let’s say that I’m implementing bank account functionality and I am writing code to implement the following:

Given a bank account
When I deposit money into the account
Then the balance should increase by the amount of the deposit

The old way

I used to write them like this:

[TestFixture]
public class When_I_deposit_money_into_the_account : Specification
{
    private Account _account;
 
    public void Establish_context()
    {
        _account = Given_a_bank_account();
    }
 
    public void Because_of()
    {
        _account.Deposit(10m);
    }
 
    [Test]
    public void Then_the_balance_should_increase_by_the_amount_of_the_deposit()
    {
        _account.Balance.ShouldEqual(10m);
    }
 
    private Account Given_a_bank_account()
    {
        return new Account();
    }
}

This style is used by RSpec and many people in the .NET community. I’ve had a lot of success doing it this way, but I’ve always had some complaints.

  • The Given/When/Then phrases are spread out and aren’t in order.
  • We have the Specification base class, which isn’t bad but I think it might confuse new people who know NUnit but don’t know my special base class.
  • If you have a lot of “given” statements, you’re tempted to use inheritance or nested contexts, each with their own level of setup, virtual properties and methods, etc. This is very unreadable and gets unwieldy very quickly.

The new way

This isn’t really a new way, but it’s new to me. I’ve seen it before over the years but I didn’t really start doing it until I joined my current team where they did it this way and I’ve come to like it a lot better than the old way.

[TestFixture]
public class Bank_account_tests
{
    [Test]
    public void Deposit()
    {
        Given_a_bank_account();
        When_I_deposit_money_into_the_account();
        Then_the_balance_should_increase_by_the_amount_of_the_deposit();
    }
 
#region Helper methods  
 
    private void Given_a_bank_account()
    {
        _account = new Account();
    }
 
    private void When_I_deposit_money_into_the_account()
    {
        _account.Deposit(10m);
    }
 
    private void Then_the_balance_should_increase_by_the_amount_of_the_deposit()
    {
        _account.Balance.ShouldEqual(10m);
    }
 
#endregion
}

The best thing about doing it this way is that the business functionality is clearly specified at the top of the class and the Given/When/Then statements aren’t spread out all over the place. There are no crazy inheritance hierarchies, base classes, or big setup methods. When I write tests, I just write out the Given/When/Then scenarios in plain text and then use Specs2Tests to generate the test code for me. Then all I have to do is fill in the private helper methods. This is really easy, like filling in the blanks.

Also, I typically hate regions but in this case I find that they work quite well because they hide the helper methods that I typically don’t want to see when I open the class file.

It depends

Obviously there are situations where this is all overkill and you can just write simple tests without Given/When/Then methods all over the place. Just do whatever makes sense. I’ve found that this new-to-me way leads to very readable and easy to maintain tests.

Why do we group our tests by file?

Posted on June 27th, 2011 in .NET, BDD, Cucumber, Ruby, TDD, unit testing by Jon Kruger

Most people I know put their unit tests in files that mirror the folder structure and filename of the actual class that is being tested. So if I have app/models/order.rb, I’ll have spec/models/order_spec.rb. The tests in the order_spec.rb file will test the code in the order.rb file. Every project I have been on, whether Ruby or .NET, has done it this way.

But have you ever thought about why we do it this way?

We do it this way on our project and I keep running into two problems. The first problem is when I am about to modify some existing code and I want to know what tests will test that portion of the code. The first place I look is the corresponding test file (based on the folder structure or filename), but that doesn’t always give me all of the tests for that functionality.

The second problem is when I refactor some existing code and the refactoring spans multiple classes. Now I have broken tests all over the place and it’s hard to reconstruct the tests so that they test the same business concepts that they were testing before. Often times the tests were testing a portion of the system of classes that I am refactoring, but were not as encompassing as they should’ve been.

Here’s the thing — when I write tests, I’m typically using the Given/When/Then style of writing tests, even in unit tests using frameworks like RSpec. I’ll have test code that looks like this:


describe "When completing an order" do
  it "should set the status to In Process" do
     # test code here
  end
end

That code tests functionality having to do with orders and is going to help me ensure that my code performs some business function correctly. But if you look at that code snippet, you don’t know what classes I’m testing. Yeah I know, it’s just an example and I left out those details. But the point is that it doesn’t matter what classes I’m testing. What matters is that my tests are testing that my code performs a certain business function.

That being said, why are we grouping our tests by file? Wouldn’t it make much more sense to group our tests by business function instead?

I already have ways to find tests for a given class. I can search my code for the class name, or if I’m in .NET I can do Find Usages and have a little window pop up that tells me everywhere a class (or method) is used.

If my classes were grouped in folders by business function instead, I get the following benefits:

  • I can see what tests exist for a given business function
  • It encourages me to write tests that test business functionality, not test data structures (classes, methods, etc.)
  • I can put all kinds of tests in there (unit tests, Cucumber tests, even manual test plans) — all in one place, all checked into source control
  • My tests document business functionality instead of documenting a class

Remember, code constructs like classes and methods are just a means to an end, our goal is to write software that provides business value and performs specific functions. So I might as well organize my tests accordingly.

(Disclaimer: I have never actually tried organizing tests this way. It makes sense to me and I think it would work great, but I might try it and find out that it doesn’t work. But if anything, maybe I’ll start a good discussion.)

Testable, Object Oriented JavaScript - slides, code, and links

Posted on May 6th, 2011 in JavaScript, Speaking, TDD, unit testing by Jon Kruger

Here are the slides, code, and links from my Testable, Object Oriented JavaScript talk at Stir Trek.

Slides
Code
JSView
Jasmine

Have fun with it, I’m always open for suggestions with this stuff, so let me know if find ways to improve it.

TDD Immersion - 1/2 day of TDD in .NET, for free!

Posted on February 8th, 2011 in TDD by Jon Kruger

TDD Immersion is a half-day session where we’ll cover what you need to know to do TDD on real .NET projects. We’ll go over stuff like:

  • How to write tests first
  • How to refactor your code to make it easier to test
  • Mocking frameworks like Rhino Mocks
  • Dependency injection - what it is, how it works, and how to set it up in your project
  • TDD tips and tricks

We’ll also walk through a sample ASP.NET MVC web app and show you how you might set up an actual project, work with ORMs and data access layers, set up your test projects, and make things easy to test.

This will take place on April 21 from 8:30-12 at the Microsoft office on Polaris Parkway in Columbus. We’ll have food and drinks for breakfast, so show up a little early to get something to eat. We’ll start promptly at 8:30.

If you plan on coming I need you to register so that we make sure that we have enough room and food for everyone.

We won’t be doing any hands-on coding this time, so you don’t need to bring your laptops. There is no wireless access at the Microsoft office.

Slides from my TDD in Action talk at CONDG

Posted on January 31st, 2011 in TDD by Jon Kruger

Since some of you asked, here are the slides from my TDD in Action talk at CONDG last week.

I’m speaking on TDD at CONDG on 1/27

Posted on January 18th, 2011 in .NET, Speaking, TDD by Jon Kruger

I’ll be speaking on test-driven development at CONDG on Thursday, Jan. 27 at 6pm. I’ll show you some live TDD coding and explain how TDD can help you write awesome code and eliminate defects. Hope to see you there!

Using Cucumber for unit tests… why not?

Posted on December 13th, 2010 in BDD, Cucumber, Ruby, TDD, unit testing by Jon Kruger

It seems that the accepted way to test in Ruby is to use Rspec for unit tests and to use Cucumber for acceptance tests (higher level functional testing). After doing a little bit of Cucumber, I’ve started to fall in love with the format of Cucumber tests.

Most Rubyists would probably agree that behavior-driven development is good (in other words, writing tests in a Given/When/Then format). We obviously do this in Cucumber (there isn’t much choice), but I’ve also written tests in this format in Rspec and in .NET.

I like BDD for two main reasons. First, I believe that software development is a series of translations. I want to translate business requirements into readable, executable specifications, then translate that into tests, then translate that into implementation code. Second, before I implement a feature and even before I write my tests, I try to write out what I want the code to do in English. If I can’t write out what I want to do in English, how and I supposed to know what I’m supposed to write in code?

Here’s my theory: if we agree that BDD is good, why don’t we write our unit tests in a format that is more amenable to BDD, that being the Cucumber format of tests? I’m not saying that we write acceptance level tests instead of unit tests, I’m saying that maybe we should write unit tests in a different format. Not only that, Cucumber tables give us a nice way to write more readable, data-driven tests. Here are a couple examples from the supermarket pricing kata (in Rspec and Cucumber).

Cucumber:

Feature: Checkout
 
  Scenario Outline: Checking out individual items
    Given that I have not checked anything out
    When I check out item <item>
    Then the total price should be the <unit price> of that item
 
  Examples:
    | item | unit price |
    | "A"  | 50         |
    | "B"  | 30         |
    | "C"  | 20         |
    | "D"  | 15         |
 
  Scenario Outline: Checking out multiple items
    Given that I have not checked anything out
    When I check out <multiple items>
    Then the total price should be the <expected total price> of those items
 
  Examples:
    | multiple items | expected total price | notes                |
    | "AAA"          | 130                  | 3 for 130            |
    | "BB"           | 45                   | 2 for 45             |
    | "CCC"          | 60                   |                      |
    | "DDD"          | 45                   |                      |
    | "BBB"          | 75                   | (2 for 45) + 30      |
    | "BABBAA"       | 205                  | order doesn't matter |
    | ""             | 0                    |                      |
 
  Scenario Outline: Rounding money
    When rounding "<amount>" to the nearest penny
    Then it should round it using midpoint rounding to "<rounded amount>"
 
    Examples:
      | amount | rounded amount |
      | 1      | 1              |
      | 1.225  | 1.23           |
      | 1.2251 | 1.23           |
      | 1.2249 | 1.22           |
      | 1.22   | 1.22           |

Rspec:

require 'spec_helper'
 
describe "Given that I have not checked anything out" do
  before :each do
    @check_out = CheckOut.new
  end
 
  [["A", 50], ["B", 30], ["C", 20], ["D", 15]].each do |item, unit_price|
  describe "When I check out an invididual item" do
    it "The total price should be the unit price of that item" do
      @check_out.scan(item)
      @check_out.total.should == unit_price
    end
  end
end
 
  [["AAA", 130], # 3 for 130
    ["BB", 45],  # 2 for 45
    ["CCC", 60],
    ["DDD", 45],
    ["BBB", 75], # (2 for 45) + 30
    ["BABBAA", 205], # order doesn't matter
    ["", 0]].each do |items, expected_total_price|
    describe "When I check out multiple items" do
      it "The total price should be the expected total price of those items" do
        individual_items = items.split(//)
        individual_items.each { |item| @check_out.scan(item) }
        @check_out.total.should == expected_total_price
      end
    end
  end
end
 
class RoundingTester
  include Rounding
end
 
[[1, 1],
  [1.225, 1.23],
  [1.2251, 1.23],
  [1.2249, 1.22],
  [1.22, 1.22]].each do |amount, rounded_amount|
  describe "When rounding an amount of money to the nearest penny" do
    it "Should round the amount using midpoint rounding" do
      RoundingTester.new.round_money(amount).should == rounded_amount
    end
  end
end

A couple things stand out to me when you compare these two. First, if I want to run data-driven tests with different values, the Cucumber syntax is so much cleaner and more descriptive. Second, the “Given I have not checked anything out” section in the Rspec version is really long and contains two nested “describe” sections (many times you end up with many more than this). When you nest sections like this, it’s really hard to see the context of things or read the tests because the “Given” text is nowhere near the nested “When” sections in the code.

Rspec follows in the footsteps of previous unit testing frameworks that write test methods in test classes (or in the case of Rspec, something that resembles test classes and methods. But is this the best way, or just the way that we’re used to? We have been writing unit tests this way for years and years because we had no other choice. But that doesn’t mean that it’s the best way.

Here are the benefits I see of using the Cucumber syntax over Rspec:

  • The tests are much easier to read (especially when doing data-driven “scenario outline” tests).
  • The Given/When/Then text is all in one place (as opposed to spread out and mixed in with code).
  • It forces me to be able to write out in English what I want the code to do.
  • Any step definition that I write can easily be reused anywhere in any other Cucumber test.
  • The code just looks cleaner. I’ve seen a lot of messy Rspec tests.
  • Rspec doesn’t have a method that corresponds to the “When” step (unless I’m missing something), so you have to shoehorn it into before(:each) or the “it” method. (I’m not sure why this is, we figured this out in the .NET world long ago.)

To be fair, there are more BDD-friendly flavors of Rspec (like rspec-given). This helps you write tests in Given/When/Then format, but I still feel like all of the underscores and symbols and syntax is getting in the way of the actual test verbiage.

Favoring Cucumber is my personal preference and I know that there are some people that would probably disagree with my opinion on this, and that’s fine. But I’m really enjoying what Cucumber brings to the table, both in terms of functionality and the syntax.

JSView: Readable, object-oriented JavaScript TDD

Posted on October 25th, 2010 in JavaScript, TDD by Jon Kruger

If you’ve tried to write JavaScript using TDD and your JavaScript deals with DOM elements, I’m guessing you’ve felt some pain. The toughest thing about writing tests against JavaScript is separating the HTML elements and the page from the JavaScript code. In the past when I’ve done this, I’ve created JavaScript classes and either passed UI elements into the constructor or by calling a method. So in the real page, I’ll have code that looks like this:

$(document).ready(function()
{
    var myClass = new MyClass(document.getElementById('blah'));
});

Then in my test class, I do something like this:

module("MyClass tests");
 
test("Test something", function()
{
    var element = new Object();
    var myClass = new MyClass(element);
});

This works fine… since JavaScript is a dynamic language, I can put any property on the “element” variable and then write tests against it, checking various properties that I know will get set on the real element (like “innerHTML”, for example).

Here’s the problem with this: if you’re writing tests first, you’re not sure what kind of elements you are going to have in the UI. Maybe “innerHTML” will get set on my element, or maybe “value”, or maybe something else. I just don’t know because I haven’t written the UI yet.

What I need it something that wraps the HTML element so that I can just call a function like getValue() and get the value, which may come from “value” in some cases, “innerHTML” in other cases, etc., depending on what kind of UI element I have.

jQuery to the rescue!

It turns out that we already have something that will wrap UI elements – jQuery. Using jQuery, we can search for elements by type (e.g. DIV, A, TR, etc.), class, or id. This works something like this:

var elements = $(’.MyCssClass’);

This returns all elements that have the MyCssClass class applied to them. But it doesn’t return the actual elements, it returns a wrapper around the HTML element. It also turns out that said wrapper has functions like val(), html(), and text() that I can use to get the values! Just what I needed.

But I still have a problem. If I’m trying to test code that deals with 20 different UI elements, that means that I am going to have to inject 20 UI elements or test doubles into my JavaScript code. This is both ugly and painful.

A simpler abstraction

JSView is an abstraction layer that sits between your JavaScript code and jQuery/HTML so that you can write tests for your JavaScript code without having the actual page and the DOM available in a test. Instead of calling jQuery methods directly, you will call methods that will interact with jQuery in your app, but will interact with a fake replacement for jQuery in your tests. Not only that, JSView will generate methods for you so that you can easily access values and events in a readable fashion. This will allow you to truly TDD your object-oriented JavaScript before the actual page even exists!

Now I can write object-oriented JavaScript classes that look something like this:

function Calculator(element, view)
{
    if (view == null)
        view = new jQueryView('Calculator', element);
 
    registerObjectProperties(this, view, ['FirstValue', 'SecondValue', 'Result', 'AddButton']);
 
    this.whenAddButtonIsClicked(function()
    {
        this.addValues();
    });
 
    this.addValues = function()
    {
        this.setResult(Number(this.getFirstValue()) + Number(this.getSecondValue()));
    }
}

Now I can write JavaScript unit tests that look like this (and without the DOM in place!):

module('When the Add button is clicked',
{
    setup: function()
    {
        calculator = new Calculator(null, new ViewTestDouble());
        calculator.setFirstValue(2);
        calculator.setSecondValue(3.5);
        calculator.clickAddButton();
    }
});
 
test('Then it should add the values in the first two textboxes and put the result in the third textbox', function()
{
    equals(calculator.getResult(), 5.5);
});

More details can be found here. The source code is up on github. I’ve been using and refining this for two years and I’m really liking how it’s turned out. I’m very interested in any feedback that you all might have, I’ve done a lot of JavaScript over the years but I’m not quite the JavaScript ninja that some other people are.

So get busy and write some tests!

I’m speaking on TDD tomorrow at Agile Lunchbox

Posted on September 28th, 2010 in Speaking, TDD by Jon Kruger

I’m speaking on test-driven development at Agile Lunchbox Wednesday afternoon. The meeting is at the Quick Solutions office (440 Polaris Pkwy, Suite 500, Westerville) at 11:45pm and food is provided.

This will be a more business-focused version of the talk. There will be some code, but this is more about why we do TDD and behavior-driven development, the benefits of TDD, the theory behind it, etc. So bring your managers and PMs along too!

A TDD success story

Posted on August 1st, 2010 in Agile, TDD by Jon Kruger

In a month or so, my co-worker and I will be wrapping up the project that we have been working on for the last 15 months. It’s a website for a company in the construction industry that bids on jobs and then tracks the progress of the jobs, purchase orders, billing, and everything else they need to run their business. We used ASP.NET MVC, Fluent NHibernate, AutoMapper, and SQL Server 2008.

We practiced test-driven development from day one. We wrote tests for our .NET code, tests for our JavaScript code, and tests for our SQL code. Right now we have over 14,000 tests, and I think we can break the 15,000 test barrier in the next month. I can run them all in about 5 minutes.

I cannot tell you how invaluable these tests have been. First of all, our application deals with money. The users will input a bunch of data about a job and then our application will tell them how much to bid on the job. We cannot afford to have bugs in our code that would miscalculate the amount to bid on a job, because that would lead either to over-bidding (in which case they would win very few) or underbidding (in which case they would win jobs and take a big hit). It just has to work.

Second, we did not have a QA team on this project, it was just the two of us developers. Frankly, I don’t have time to go back and manually test stuff in the app or regression test it when we need to make a change and deploy something. We do have bugs from time to time, but we haven’t had any critical bugs.

Because of our tests, we have been able to get as close to continuous deployment as I would feel comfortable with. On average, we deploy 2-3 times a week. When we go to do a deployment, I usually go through the site and manually test the new features that we are about to release. If I don’t find any problems, I run our deployment (which is all automated), and 5 minutes later, the changes have been deployed. I never go back and regression test old features or stuff in areas that we didn’t change.

Since the whole test and deploy process takes only about 15 minutes, users get their changes quickly. They don’t have to wait until later in the week, or until our next scheduled release. We have reduced the cost of change to pretty much just the time that it takes to code the changes.

All of this is possible because we were diligent about test-driven development and writing good unit tests for everything. The reason that we don’t spend time regression testing is that we don’t expect anything to be broken, and it very rarely ever is. This means that we can spend more time delivering business value and less time ensuring that we didn’t break something we wrote a year ago.

I’m not writing this to say how awesome we are as developers, because anyone can write tests and have the same kind of success. Also, we could’ve cheated and not written tests and ended up with potentially costly bugs and more time spent regression testing.

This is why I practice test-driven development. You end up with well-designed code, you drastically reduce bug counts, you can release more often, your codebase stays under control, and you have a lot less stress.

Next Page »