Jon Kruger

When I write unit tests, I use the BDD-style GIven/When/Then format so that the tests are descriptive and explain the business functionality that is being implemented. But I’ve recently changed the way that I do it.

Let’s say that I’m implementing bank account functionality and I am writing code to implement the following:

Given a bank account
When I deposit money into the account
Then the balance should increase by the amount of the deposit

The old way

I used to write them like this:

[TestFixture]
public class When_I_deposit_money_into_the_account : Specification
{
    private Account _account;
 
    public void Establish_context()
    {
        _account = Given_a_bank_account();
    }
 
    public void Because_of()
    {
        _account.Deposit(10m);
    }
 
    [Test]
    public void Then_the_balance_should_increase_by_the_amount_of_the_deposit()
    {
        _account.Balance.ShouldEqual(10m);
    }
 
    private Account Given_a_bank_account()
    {
        return new Account();
    }
}


This style is used by RSpec and many people in the .NET community. I’ve had a lot of success doing it this way, but I’ve always had some complaints.

• The Given/When/Then phrases are spread out and aren’t in order.
• We have the Specification base class, which isn’t bad but I think it might confuse new people who know NUnit but don’t know my special base class.
• If you have a lot of “given” statements, you’re tempted to use inheritance or nested contexts, each with their own level of setup, virtual properties and methods, etc. This is very unreadable and gets unwieldy very quickly.

The new way

This isn’t really a new way, but it’s new to me. I’ve seen it before over the years but I didn’t really start doing it until I joined my current team where they did it this way and I’ve come to like it a lot better than the old way.

[TestFixture]
public class Bank_account_tests
{
    [Test]
    public void Deposit()
    {
        Given_a_bank_account();
        When_I_deposit_money_into_the_account();
        Then_the_balance_should_increase_by_the_amount_of_the_deposit();
    }
 
#region Helper methods  
 
    private void Given_a_bank_account()
    {
        _account = new Account();
    }
 
    private void When_I_deposit_money_into_the_account()
    {
        _account.Deposit(10m);
    }
 
    private void Then_the_balance_should_increase_by_the_amount_of_the_deposit()
    {
        _account.Balance.ShouldEqual(10m);
    }
 
#endregion
}


The best thing about doing it this way is that the business functionality is clearly specified at the top of the class and the Given/When/Then statements aren’t spread out all over the place. There are no crazy inheritance hierarchies, base classes, or big setup methods. When I write tests, I just write out the Given/When/Then scenarios in plain text and then use Specs2Tests to generate the test code for me. Then all I have to do is fill in the private helper methods. This is really easy, like filling in the blanks.

Also, I typically hate regions but in this case I find that they work quite well because they hide the helper methods that I typically don’t want to see when I open the class file.

It depends

Obviously there are situations where this is all overkill and you can just write simple tests without Given/When/Then methods all over the place. Just do whatever makes sense. I’ve found that this new-to-me way leads to very readable and easy to maintain tests.

• Leave a comment... (4)

Most people I know put their unit tests in files that mirror the folder structure and filename of the actual class that is being tested. So if I have app/models/order.rb, I’ll have spec/models/order_spec.rb. The tests in the order_spec.rb file will test the code in the order.rb file. Every project I have been on, whether Ruby or .NET, has done it this way.

But have you ever thought about why we do it this way?

We do it this way on our project and I keep running into two problems. The first problem is when I am about to modify some existing code and I want to know what tests will test that portion of the code. The first place I look is the corresponding test file (based on the folder structure or filename), but that doesn’t always give me all of the tests for that functionality.

The second problem is when I refactor some existing code and the refactoring spans multiple classes. Now I have broken tests all over the place and it’s hard to reconstruct the tests so that they test the same business concepts that they were testing before. Often times the tests were testing a portion of the system of classes that I am refactoring, but were not as encompassing as they should’ve been.

Here’s the thing — when I write tests, I’m typically using the Given/When/Then style of writing tests, even in unit tests using frameworks like RSpec. I’ll have test code that looks like this:


describe "When completing an order" do
  it "should set the status to In Process" do
     # test code here
  end
end


That code tests functionality having to do with orders and is going to help me ensure that my code performs some business function correctly. But if you look at that code snippet, you don’t know what classes I’m testing. Yeah I know, it’s just an example and I left out those details. But the point is that it doesn’t matter what classes I’m testing. What matters is that my tests are testing that my code performs a certain business function.

That being said, why are we grouping our tests by file? Wouldn’t it make much more sense to group our tests by business function instead?

I already have ways to find tests for a given class. I can search my code for the class name, or if I’m in .NET I can do Find Usages and have a little window pop up that tells me everywhere a class (or method) is used.

If my classes were grouped in folders by business function instead, I get the following benefits:

• I can see what tests exist for a given business function
• It encourages me to write tests that test business functionality, not test data structures (classes, methods, etc.)
• I can put all kinds of tests in there (unit tests, Cucumber tests, even manual test plans) — all in one place, all checked into source control
• My tests document business functionality instead of documenting a class

Remember, code constructs like classes and methods are just a means to an end, our goal is to write software that provides business value and performs specific functions. So I might as well organize my tests accordingly.

(Disclaimer: I have never actually tried organizing tests this way. It makes sense to me and I think it would work great, but I might try it and find out that it doesn’t work. But if anything, maybe I’ll start a good discussion.)

• Leave a comment... (3)

Here are the slides, code, and links from my Testable, Object Oriented JavaScript talk at Stir Trek.

Slides
Code
JSView
Jasmine

Have fun with it, I’m always open for suggestions with this stuff, so let me know if find ways to improve it.

• Leave a comment... (0)

It seems that the accepted way to test in Ruby is to use Rspec for unit tests and to use Cucumber for acceptance tests (higher level functional testing). After doing a little bit of Cucumber, I’ve started to fall in love with the format of Cucumber tests.

Most Rubyists would probably agree that behavior-driven development is good (in other words, writing tests in a Given/When/Then format). We obviously do this in Cucumber (there isn’t much choice), but I’ve also written tests in this format in Rspec and in .NET.

I like BDD for two main reasons. First, I believe that software development is a series of translations. I want to translate business requirements into readable, executable specifications, then translate that into tests, then translate that into implementation code. Second, before I implement a feature and even before I write my tests, I try to write out what I want the code to do in English. If I can’t write out what I want to do in English, how and I supposed to know what I’m supposed to write in code?

Here’s my theory: if we agree that BDD is good, why don’t we write our unit tests in a format that is more amenable to BDD, that being the Cucumber format of tests? I’m not saying that we write acceptance level tests instead of unit tests, I’m saying that maybe we should write unit tests in a different format. Not only that, Cucumber tables give us a nice way to write more readable, data-driven tests. Here are a couple examples from the supermarket pricing kata (in Rspec and Cucumber).

Cucumber:

Feature: Checkout
 
  Scenario Outline: Checking out individual items
    Given that I have not checked anything out
    When I check out item <item>
    Then the total price should be the <unit price> of that item
 
  Examples:
    | item | unit price |
    | "A"  | 50         |
    | "B"  | 30         |
    | "C"  | 20         |
    | "D"  | 15         |
 
  Scenario Outline: Checking out multiple items
    Given that I have not checked anything out
    When I check out <multiple items>
    Then the total price should be the <expected total price> of those items
 
  Examples:
    | multiple items | expected total price | notes                |
    | "AAA"          | 130                  | 3 for 130            |
    | "BB"           | 45                   | 2 for 45             |
    | "CCC"          | 60                   |                      |
    | "DDD"          | 45                   |                      |
    | "BBB"          | 75                   | (2 for 45) + 30      |
    | "BABBAA"       | 205                  | order doesn't matter |
    | ""             | 0                    |                      |
 
  Scenario Outline: Rounding money
    When rounding "<amount>" to the nearest penny
    Then it should round it using midpoint rounding to "<rounded amount>"
 
    Examples:
      | amount | rounded amount |
      | 1      | 1              |
      | 1.225  | 1.23           |
      | 1.2251 | 1.23           |
      | 1.2249 | 1.22           |
      | 1.22   | 1.22           |


Rspec:

require 'spec_helper'
 
describe "Given that I have not checked anything out" do
  before :each do
    @check_out = CheckOut.new
  end
 
  [["A", 50], ["B", 30], ["C", 20], ["D", 15]].each do |item, unit_price|
  describe "When I check out an invididual item" do
    it "The total price should be the unit price of that item" do
      @check_out.scan(item)
      @check_out.total.should == unit_price
    end
  end
end
 
  [["AAA", 130], # 3 for 130
    ["BB", 45],  # 2 for 45
    ["CCC", 60],
    ["DDD", 45],
    ["BBB", 75], # (2 for 45) + 30
    ["BABBAA", 205], # order doesn't matter
    ["", 0]].each do |items, expected_total_price|
    describe "When I check out multiple items" do
      it "The total price should be the expected total price of those items" do
        individual_items = items.split(//)
        individual_items.each { |item| @check_out.scan(item) }
        @check_out.total.should == expected_total_price
      end
    end
  end
end
 
class RoundingTester
  include Rounding
end
 
[[1, 1],
  [1.225, 1.23],
  [1.2251, 1.23],
  [1.2249, 1.22],
  [1.22, 1.22]].each do |amount, rounded_amount|
  describe "When rounding an amount of money to the nearest penny" do
    it "Should round the amount using midpoint rounding" do
      RoundingTester.new.round_money(amount).should == rounded_amount
    end
  end
end


A couple things stand out to me when you compare these two. First, if I want to run data-driven tests with different values, the Cucumber syntax is so much cleaner and more descriptive. Second, the “Given I have not checked anything out” section in the Rspec version is really long and contains two nested “describe” sections (many times you end up with many more than this). When you nest sections like this, it’s really hard to see the context of things or read the tests because the “Given” text is nowhere near the nested “When” sections in the code.

Rspec follows in the footsteps of previous unit testing frameworks that write test methods in test classes (or in the case of Rspec, something that resembles test classes and methods. But is this the best way, or just the way that we’re used to? We have been writing unit tests this way for years and years because we had no other choice. But that doesn’t mean that it’s the best way.

Here are the benefits I see of using the Cucumber syntax over Rspec:

• The tests are much easier to read (especially when doing data-driven “scenario outline” tests).
• The Given/When/Then text is all in one place (as opposed to spread out and mixed in with code).
• It forces me to be able to write out in English what I want the code to do.
• Any step definition that I write can easily be reused anywhere in any other Cucumber test.
• The code just looks cleaner. I’ve seen a lot of messy Rspec tests.
• Rspec doesn’t have a method that corresponds to the “When” step (unless I’m missing something), so you have to shoehorn it into before(:each) or the “it” method. (I’m not sure why this is, we figured this out in the .NET world long ago.)

To be fair, there are more BDD-friendly flavors of Rspec (like rspec-given). This helps you write tests in Given/When/Then format, but I still feel like all of the underscores and symbols and syntax is getting in the way of the actual test verbiage.

Favoring Cucumber is my personal preference and I know that there are some people that would probably disagree with my opinion on this, and that’s fine. But I’m really enjoying what Cucumber brings to the table, both in terms of functionality and the syntax.

• Leave a comment... (4)

I had a simple problem today. I had to turn this:

… into this:


[TestFixture]
public class When_depositing_money_into_an_account : Specification
{
    [Test]
    public void should_add_the_specified_amount_into_the_account()
    {
 
    }
}
 
[TestFixture]
public class When_withdrawing_money_from_an_account : Specification
{
    [Test]
    public void should_withdraw_the_specified_amount_from_the_account()
    {
 
    }
 
    [Test]
    public void should_specify_that_the_withdrawal_was_successful()
    {
 
    }
}


I really didn’t want to do all of this typing every time, so I wrote code to do it for me. It works like this:

1) Copy the acceptance criteria text onto the clipboard
2) Run Specs2Tests.exe (will put the code onto the clipboard for you)
3) Paste the code into your code file

Now I can get all of my acceptance criteria nailed down in a text file where I don’t have the ceremony of a programming language, then let something convert them into tests. Since I just cranked this out today, I’m guessing that it doesn’t handle every scenario that you might think of, and it only writes out C# code. There is no reason that it couldn’t spit out VB.NET, RSpec, or anything else. If it doesn’t do what you want, get on github, create a fork, and send me a patch!

The code is here: http://github.com/jonkruger/specs2tests. Have fun.

• Leave a comment... (0)

I wanted to come up with a way to show people how to use Rhino Mocks (other than telling them to read the documentation). What better way to do this than by showing you how it works through a bunch of simple unit tests that document how Rhino Mocks works?

So that’s what I did. You can view the code here, or if you want to download the whole project and run the tests, you can get the whole thing here.

(If you’re interested in Moq, how it compares to Rhino Mocks, and to see Moq documented through tests, check out Steve Horn’s post.)

UPDATE: Fixed the test that was incorrectly showing how to use Expect() and VerifyAllExpectations(). Thanks to Sharon for pointing this out.

• Leave a comment... (8)

Recently I had the privilege of hearing Uncle Bob Martin talk at the Columbus Ruby Brigade. Among the many nuggets of wisdom that I learned that night, my favorite part was the Automated Testing Triangle. I don’t know if Uncle Bob made this up or if he got it from somewhere else, but it goes something like this.

At the bottom of the triangle we have unit tests. These tests are testing code, individual methods in classes, really small pieces of functionality. We mock out dependencies in these tests so that we can test individual methods in isolation. These tests are written using testing frameworks like NUnit and use mocking frameworks like Rhino Mocks. Writing these kinds of tests will help us prove that our code is working and it will help us design our code. They will ensure that we only write enough code to make our tests pass. Unit tests are the foundation of a maintainable codebase.

But there will be situations where unit tests don’t do enough for us because we will need to test multiple parts of the system working together. This means that we need to write integration tests — tests that test the integration between different parts of the system. The most common type of integration test is a test that interacts with the database. These tests tend to be slower and are more brittle, but they serve a purpose by testing things that we can test with unit tests.

Everything we’ve discussed so far will test technical behavior, but doesn’t necessarily test functional business specifications. At some point we might want to write tests that read like our technical specs so that we can show that our code is doing what the business wants it to do. This is when we write acceptance tests. These tests are written using tools like Cucumber, Fitnesse, StoryTeller, and NBehave. These tests are usually written in plain text sentences that a business analyst could write, like this:
As a user
When I enter a valid username and password and click Submit
Then I should be logged in


At this point, we’re are no longer just testing technical aspects of our system, we are testing that our system meets the functional specifications provided by the business.

By now we should be able to prove that our individual pieces of code are working, that everything works together, and that it does what the business wants it to do — and all of it is automated. Now comes the manual testing. This is for all of the random stuff — checking to make sure that the page looks right, that fancy AJAX stuff works, that the app is fast enough. This is where you try to break the app, hack it, put weird values in, etc.

I find that the testing triangle on most projects tends to look more like this triangle. There are some automated integration tests, but these tests don’t use mocking frameworks to isolate dependencies, so they are slow and brittle, which makes them less valuable. An enormous amount of manpower is spent on manual testing.

Lots of projects are run this way, and many of them are successful. So what’s the big deal? Becuase what really matters is the total cost of ownership of an application over the entire lifetime of the application. Most applications need to be changed quite often, so there is much value in doing things that will allow the application be changed easily and quickly.

Many people get hung up on things like, “I don’t have time to write tests!” This is a short term view of things. Sometimes we have deadlines that cannot be moved, so I’m not denying this reality. But realize that you are making a short term decision that will have long term effects.

If you’ve ever worked on a project that had loads of manual testing, then you can at least imagine how nice it would be to have automated tests that would test a majority of your application by clicking a button. You could deploy to production quite often because regression testing would take drastically less time.

I’m still trying to figure out how to achieve this goal. I totally buy into Uncle Bob’s testing triangle, but it requires a big shift in the way we staff teams. For example, it would really help if QA people knew how to use automated testing tools (which may require basic coding skills). Or maybe we have developers writing more automated tests (beyond the unit tests that they usually write). Either way, the benefits of automated testing are tremendous and will save loads of time and money over the life of an application.

• Leave a comment... (2)

One question that I hear from people who are new to TDD or writing tests is, “How much longer is my feature going to take if I write tests?” This is a valid question. We all have deadlines, so if we have to add something extra to our development process, it better be worth it.

I don’t think I could say it better than this post did, so I’ll just let you read it for yourself.

• Leave a comment... (0)

A lot happens from the time that a business owner envisions an idea in their mind and the time that the idea becomes software. The trick is getting through the whole software development process without losing the original idea.

Have you ever played that party game when you go around the circle whispering the same phrase and when you get to the end the phrase is completely different than when it started? Software development ends up like that a lot (especially when the business owner is changing the original idea!).

The problem is that we all speak a different languages. Developers speak one language, BAs another, PMs another, DBAs another, users another, business people another, executives another, and so on. So a large part of our job is learning how to translate what these people want into developer language (code), and doing it so that, in the end, the software speaks to them in their language.

Here’s a simple example:

Executive: “When we hire new employees, we need to make sure that they have a computer ready for them when they start.”

Business analyst: “User will enter the number of new employees on the screen. The system will check the inventory and make sure that we have enough machines, monitors, keyboards, and mice for the new employees. Each employee should have two monitors.”

Now it’s our turn. As developers, we have to do several translations in the process of writing the software. The DBA (or you) may have to design the database schema. You (the developer) have to write the code.

Remember, the goal is to not lose the original idea. That means that each time we “translate” the original idea, we need to do it little bits at a time.

Usually business analysts don’t give you specs that are written exactly how you want them. This is not a knock on BAs, but they speak a different language than us developers. What we really want from them is a set of acceptance criteria. In other words, we need to know what we have to do in order to complete the feature. Not only that, we need to know how we are going to test our feature so that we know that it’s working.

So let’s take the business analyst’s specs and translate them into acceptance criteria:

Given a stash of unused hardware
   When a user enters the number of new employees
      Then it should verify that we have one machine for each new employee
      Then it should verify that we have two monitors for each new employee
      Then it should verify that we have one keyboard for each new employee
      Then it should verify that we have one mouse for each new employee

   When we don’t have enough mice for new employees
      Then we need to order new mice so that we have one for each new employee

We’re not saying anything drastically different from what the business analyst said. But the way that we wrote it is important. Notice the use of the words given, when, then. Here’s the next translation:


public class Given_a_stash_of_unused_hardware
{
}
 
public class When_a_user_enters_the_number_of_new_employees
    : Given_a_stash_of_unused_hardware
{
    [Test]
    public void Then_it_should_verify_that_we_have_one_machine_for_each_new_employee()
    {
    }
 
    [Test]
    public void Then_it_should_verify_that_we_have_two_monitors_for_each_new_employee()
    {
    }
 
    [Test]
    public void Then_it_should_verify_that_we_have_one_keyboard_for_each_new_employee()
    {
    }
 
    [Test]
    public void Then_it_should_verify_that_we_have_one_mouse_for_each_new_employee()
    {
    }
}
 
public class When_we_don't_have_enough_machines_for_new_employees
    : Given_a_stash_of_unused_hardware
{
    [Test]
    public void Then_we_need_to_order_new_machines_so_that_we_have_one_for_each_new_employee()
    {
    }
}
 
public class When_we_don't_have_enough_monitors_for_new_employees
    : Given_a_stash_of_unused_hardware
{
    [Test]
    public void Then_we_need_to_order_new_monitors_so_that_we_have_two_for_each_new_employee()
    {
    }
}
 
public class When_we_don't_have_enough_keyboards_for_new_employees
    : Given_a_stash_of_unused_hardware
{
    [Test]
    public void Then_we_need_to_order_new_keyboards_so_that_we_have_one_for_each_new_employee()
    {
    }
}
 
public class When_we_don't_have_enough_mice_for_new_employees
    : Given_a_stash_of_unused_hardware
{
    [Test]
    public void Then_we_need_to_order_new_mice_so_that_we_have_one_for_each_new_employee()
    {
    }
}


This is how you do behavior driven development. We took our acceptance criteria and wrote them out as code in the form of unit tests. These unit tests will prove that our code is working (when we get to that point) and it will also act as our documentation of what the code is supposed to do.

The reason that this is important is that we’re still in the middle of doing our translation. A lot of people take the tech specs and immediately start writing implementation code. But by doing that, you skip a step in the translation, and when you do that, you risk losing some of the original intent of the feature. This is one of the reason why writing tests before you write implementation code is important. First of all, if you write your tests first, then when they’re passing, you know that you’re done. Second, you’ve written out what the feature is supposed to do — nothing more, nothing less. This is going to help us implement the feature without losing the original intent of the person who thought it up.

Now, yes now, you can go write the implementation code. By focusing on what the end product should be and since you wrote your tests first, this part should be easy. It’s much easier to achieve your goal when you know what the goal is!

• Leave a comment... (1)

In case you haven’t heard, the Software Engineering 101 Conference is going on in Columbus on September 23. This is a one-day event where you will learn about software design topics and techniques such as object-oriented programming and the SOLID software design principles as well as a super-special hands-on test driven development session! This is all good stuff that every developer should know and it will be well worth your time. Rather than recount all the details, I’ll let you read more on Jim’s blog.

This is a FREE event, so all you have to do is ask your employer if you can go, they don’t even have to pay anything for it. I feel like I say this a lot, but these skills are essential for any software developer and have revolutionized the way that I write code.

Registration is limited, and I expect that it will fill up relatively soon, so don’t wait! You can click here to register.

• Leave a comment... (0)