Jon Kruger -
  • About Me
  • Blog
  • Values
  • Presentations
About Me
Blog
Values
Presentations
  • About Me
  • Blog
  • Values
  • Presentations
Jon Kruger
JavaScript, unit testing

Running QUnit tests using continuous integration

On my current project we will be using QUnit to writing tests for our JavaScript.  What would be really nice is if our CI build could run JavaScript unit tests.

Well, it can!  It only takes a few magic potions and sacrificing a few small animals.

Luckily someone had done this before and blogged about it so I pretty much just had to follow instructions.  Here’s the long and short of it:

  • We had to get WatiN, which will be used to create an instance of Internet Explorer that will run our HTML page with the QUnit tests in it.
  • We had to get the IterativeTest addin for NUnit.  This will create one NUnit test for each QUnit test.

In the code, I added an IterativeTest (a test decorated with the [IterativeTest] attribute) that will run all of the QUnit tests.  Since we all use ReSharper and ReSharper doesn’t recognize the [IterativeTest] attribute, I also added a normal NUnit [Test] that will run all of the QUnit tests inside one test and fail if any of the QUnit tests fail.  So we should have all of our bases covered.

February 11, 2009by Jon Kruger
JavaScript, TDD, unit testing

Code & slides from Unit Testing JavaScript talk

Here are the slides and code from the Unit Testing JavaScript talk. Or if you one of those people you can get it all from the Subversion repository at http://minisudoku.googlecode.com/svn/trunk/.

(UPDATE 2/25/09: these links point to the latest versions of the slides and the code, which now include examples using both JSSpec and QUnit.)

November 25, 2008by Jon Kruger
JavaScript, TDD, unit testing

Unit Testing JavaScript – Test Driven Development in action!

So far I’ve talked about JavaScript basics, JavaScript test frameworks, and writing testable code. Now for the real fun… let’s do some test driven development!

I know that TDD seems somewhat backwards to a lot of people, and it’s hard to undo the years and years of not writing tests. I learned from watching people doing it and learning the thought process and reprogramming my head so that I could think in a new way. So hopefully this will help you understand the thought process.

Step 1 – Get your requirements

So you’re about to start on a new feature or fix a bug. Either way, you will get requirements in some form.

In this case, we want to create a greatly slimmed down version of Sudoku. Here are the rules:

  • The puzzle will have only 4 boxes in a 2×2 grid.
  • You can enter numbers 1 through 4 in a box by selecting a box and pressing the number keys on the keyboard
  • Selecting a box and hitting the Delete key should remove the value in the box
  • You can change the numbers or remove the numbers at any time
  • Give the user a button to click that will check to see if the puzzle is valid
  • Give the user a button that will give the user a new puzzle. A new puzzle will have 2 of the numbers already filled out.
  • A puzzle is valid if:
    • Each box has a number 1 through 4
    • Each box has a different number

OK, so what do you do now? Most people would start hammering out some JavaScript classes. But we’re not going to do that, because we’re not done with the design phase yet.

What do I mean by that? Well, one of the big reasons that we write tests first is to help us design our code. Sure, you could write tests after you write the actual code. But if you do it that way, you will spend lots of time refactoring code (that you don’t have tests around yet) in order to make it testable. That’s wasted time, so we’ll do our design up front by outlining our tests, writing the tests, and then writing the code.

Step 2 – Refine the requirements

Let’s look at the first bullet point in our requirements:
The puzzle will have only 4 boxes in a 2×2 grid.

Business requirements aren’t always written out as individual acceptance criteria, so let’s split this line into individual requirements and write them out.

  • The puzzle should have 4 boxes
  • The boxes should be in a 2×2 grid

Step 3 – Write out the test methods

Now that we have that, let’s translate that into tests:


describe('When creating a new Puzzle',
{
'The Puzzle should have 4 Boxes': function()
{
}

'The boxes should be in a 2x2 grid': function()
{
}
});

Step 4 – Stub out implementation classes

Now we have two tests. These tests will help us design our code. Let’s see what we can learn from these tests:

  • We need a Puzzle object
  • We need a Box object
  • The Puzzle should have a collection of Boxes
  • We need a way to ask the Puzzle for a box at a given coordinate in the grid

Let’s write these classes and methods, but we won’t implement the methods yet.


function Puzzle()
{
var boxes = new Array();

this.getAllBoxes = function()
{
}

this.getBox = function(columnIndex, rowIndex)
{
}
}

function Box(columnIndex, rowIndex)
{
this.getColumnIndex = function()
{
return columnIndex;
}

this.getRowIndex = function()
{
return rowIndex;
}
}

Step 5 – Write test code

Now that we have our objects, we can write the code inside our test methods (I’m using JSSpec as my testing framework):


describe('When creating a new Puzzle',
{
before_each: function()
{
puzzle = new Puzzle();
},

'The Puzzle should have 4 Boxes': function()
{
value_of(puzzle.getAllBoxes().length).should_be(4);
},

'The boxes should be in a 2x2 grid': function()
{
for (var columnIndex = 0; columnIndex < 2; columnIndex++) { for (var rowIndex = 0; rowIndex < 2; rowIndex++) { var box = puzzle.getBox(columnIndex, rowIndex); value_of(box.getColumnIndex()).should_be(columnIndex); value_of(box.getRowIndex()).should_be(rowIndex); } } } });

We've written our tests. Now let's run them and watch them fail. The reason we do this is because if these tests pass now, then they're probably not written correctly and we don't want false positives.

Failing tests

Yep, they're failing, just as I expected.

Step 6 - Write implementation code

Now, yes now, you finally can write your implementation code. I know that some of you have been chomping at the bit to get to this point. So here we go.


function Puzzle()
{
var boxes = new Array();

for (var columnIndex = 0; columnIndex < 2; columnIndex++) { for (var rowIndex = 0; rowIndex < 2; rowIndex++) { var box = new Box(columnIndex, rowIndex); if (boxes[columnIndex] == undefined) boxes[columnIndex] = new Array(); boxes[columnIndex][rowIndex] = box; } } this.getAllBoxes = function() { var list = new Array(); for (var column = 0; column < boxes.length; column++) { for (var row = 0; row < boxes[column].length; row++) list.push(boxes[column][row]); } return list; } this.getBox = function(columnIndex, rowIndex) { return boxes[columnIndex][rowIndex]; } } function Box(columnIndex, rowIndex) { this.getColumnIndex = function() { return columnIndex; } this.getRowIndex = function() { return rowIndex; } }

Let's run the tests and see if the code works.

Success!

Sweet goodness. But I've only implemented one of the requirements. No problem, I'll just go back to step 1 again and go through the process with the next requirement and repeat until I'm done with all of the requirements.

Let's take a look at what we have now.

  • We have code that we know is working correctly
  • We have tests to prove that the code is working correctly, both now and from now on
  • We have tests that describe what the code is supposed to do. Our tests are documenting our code!
  • We have well designed classes
  • Our classes do only what they're supposed to do -- we didn't waste time writing things that we don't need
  • We have peace of mind!!

Let me talk about this peace of mind. This is something that I can describe to you but you can't understand it until you experience it. I am 99% sure that the code that I wrote is working now and will continue to work in the future. I can refactor my code or make changes and I know that it will be working correctly when I'm done. I can guarantee you that the only way that my code has bugs is if there is some requirement that I didn't know about. Can you say that about your code?

What happens when it's a day or two before you put something into production and they find a nasty bug that absolutely has to be fixed before you go live? (Certainly not the ideal situation, but it happens all the time.) Wouldn't you love to have a suite of tests to run so that you can be less likely to break something else when you fix the bug? (And remember, bugs are much more costly and difficult to fix when they're in production.)

Have you ever been in that endless cycle of fixing a bug only to break something else, and then when you fix that bug you break something else, and on an on?

This is why I write tests. And this is why I will never go back.

How to get started with TDD

People ask me how to get started doing TDD. The best way is to just do it. The next time you have a simple feature or a simple bug to fix, write your tests first. It takes awhile to get into the mindset, but once you do it a few times it starts to make a lot more sense. If you know someone else who is good at TDD, testing, mocking, etc., pair program with them for a day and watch how they work. You'll need to get better at separation of concerns, programming to interfaces, and the like. Maybe that will be my next post. :)

Completed sample project

I went through the entire exercise with the MiniSudoku project. You can download the finished product here. (UPDATE 2/25/09: this link contains examples using both JSSpec and QUnit.)

Good luck!

November 25, 2008by Jon Kruger
JavaScript, TDD, unit testing

Unit Testing JavaScript – Writing testable code

Last time we talked about JavaScript test frameworks. But a test framework is no good if you don’t write testable code.

Writing testable JavaScript

As is the case with .NET code, one of the most important parts of testing is writing code that you can actually test. Let’s look at some untestable JavaScript code and what we can do to make it testable.


// Untestable JavaScript :(
function MyClass
{
this.showMessage = function(message)
{
$("#messageTextBox").text(message);
}

this.anotherShowMessage = function(message)
{
document.getElementById("anotherMessageTextBox").innerText = message;
}
}

This code is untestable because it directly references UI elements in a page by name. When you run unit tests using pretty much any JavaScript testing framework, you create a separate page to run your tests from. That means that your UI elements will not be on that page, so any code that references these elements by name will not be testable.

Here’s how you would make this testable:


// Testable JavaScript!
function MyClass()
{
var element;

this.getElement = function()
{
return element;
}

this.setElement = function(newElement)
{
element = newElement;
}

this.showMessage = function(message)
{
element.innerText = message;
}
}

// unit test
describe('When showing a message',
{
'The message should be displayed in the element': function()
{
var myClass = new MyClass();
myClass.setElement(new function() { this.innerText = null; });

myClass.showMessage('I love testing!');

value_of(myClass.getElement().innerText).should_be('I love testing!');
}
});

Notice how MyClass still has the notion of a UI element, but now the UI element is injected by whoever is using the class. And since JavaScript is a dynamic language, this is easy to test because we don’t need an actual UI element in our unit test, we just need an object that has an “innerText” variable on it. Creating said object is as easy as doing this:

var myFakeUIElement = new function() { this.innerText = null; }

This is what I can do in my test. I tell MyClass that its element is my fake UI element, I call showMessage(), and then I verify that the innerText variable of my fake UI element got set.

Another example:


// Untestable JavaScript :(
function MyClass()
{
this.showMessage = function(message)
{
alert(message);
}
}

I think this one is pretty obvious. If you try and test this method, you’re going to have an alert box popping up on the screen in the middle of your tests. Not going to work. Try this instead:


// Testable JavaScript
function MyClass()
{
this.showMessage = function(message)
{
this.showAlertBox(message);
}

this.showAlertBox = function(message)
{
alert(message);
}
}

describe('When showing a message',
{
'The message should be displayed in an alert box': function()
{
var alertMessage;
var myClass = new MyClass();
myClass.showAlertBox = function(message) { alertMessage = message; }

myClass.showMessage('I love testing!');

value_of(alertMessage).should_be('I love testing!');
}
});

Here I separated my concerns again — I created a separate method that only shows the alert box. I can “mock” this method by simply redefining it to whatever I want (in this case I’m just storing what was passed into it). Now I can test that when I call MyClass.showMessage(), I can verify that it called the method that would show an alert box.

Time for more sadness:


// Untestable JavaScript :(
function MyClass()
{
this.result = null;
this.loadCustomerData = function(customerId)
{
$.ajax({
type: "POST",
url: "CustomerService.asmx/LoadCustomer",
data: "customerId=" + customerId,
success: function(msg)
{
this.result = msg.d; // return value from web service
}
});
}
}

This is an asynchronous Ajax call using jQuery. The problem here is that because this is actually making an asychronous Ajax call, the method will return and our test code will run before the callback executes.

There are two ways to solve this:

Solution 1:

// Testable JavaScript
function MyClass()
{
this.result = null;
this.loadCustomerData = function(customerId)
{
$.ajax({
async: false,
type: "POST",
url: "CustomerService.asmx/LoadCustomer",
data: "customerId=" + customerId,
success: function(msg)
{
this.result = msg.d; // return value from web service
}
});
}
}

describe('When loading a Customer',
{
'The customer name should be returned': function()
{
var alertMessage;
var myClass = new MyClass();
myClass.loadCustomerData(1);
value_of(myClass.result).should_be('Jon');
}
});

What changed here? I added “async: false” to the $.ajax() call. Now jQuery will run this method synchronously and I will be able to test it.

Caveat: This violates the basic rules of “what is a unit test” because an external dependency is involved (in this case, my .asmx web service). You can decide for yourself whether this is bad or not. In my case, the test still runs lightning fast because all it does it return some random numbers and it never hits a database, so it doesn’t bother me too much. But if you have a more complicated web service that you’re running and it takes a long time to return data, you may not want to do it this way. Here’s another way to fix this:

Solution 2:

function MyClass()
{
this.result = null;
this.loadCustomerData = function(customerId)
{
var myClass = this;
this.makeAjaxCall(
"POST",
"CustomerService.asmx/LoadCustomer",
"customerId=" + customerId,
function(ajaxResult) { myClass.result = ajaxResult; });
}

this.makeAjaxCall = function(type, url, data, successCallback)
{
var result;
$.ajax({
type: type,
url: url,
data: data,
success: function(msg)
{
successCallback(msg.d); // return value from web service
}
});
return result;
}
}

describe('When loading a Customer',
{
'The customer name should be returned': function()
{
var ajaxType;
var ajaxUrl;
var ajaxData;
var ajaxSuccessCallback;

var alertMessage;
var myClass = new MyClass();
myClass.makeAjaxCall = function(type, url, data, successCallback)
{
ajaxType = type;
ajaxUrl = url;
ajaxData = data;
ajaxSuccessCallback = successCallback;
};

myClass.loadCustomerData(1);
value_of(ajaxType).should_be('POST');
value_of(ajaxUrl).should_be('CustomerService.asmx/LoadCustomer');
value_of(ajaxData).should_be('customerId=1');

ajaxSuccessCallback('somevalue');
value_of(myClass.result).should_be('somevalue');
}
});

Slightly more complicated doing it this way, but if the situation warrants it, it’s worth it.

Next stop: we’ll do some test driven development and see this stuff in action!

November 25, 2008by Jon Kruger
JavaScript, TDD, unit testing

Unit Testing JavaScript – JavaScript test frameworks

In my last post I gave you some basic information on JavaScript that will help you get started on the road to testing your JavaScript. Now for the JavaScript test frameworks.

JavaScript test frameworks

I did some research and there are quite a few JavaScript test frameworks out there:

JSSpec J3Unit
QUnit JSUnit
YUI Test Screw-Unit
JSNUnit script.aculo.us unit testing
TestCase Crosscheck
RhinoUnit jqUnit

So apparently there are people that are trying to solve this problem, but how do you choose between all of these frameworks?

When I want to determine the relevance of something, I go straight to search.twitter.com. I figure that if no one is talking about a JavaScript framework on Twitter or if something is getting a lot of negative comments, then it’s probably not worth looking into.

Using my completely unscientific Measure of Relevance I was able to narrow down my list of JavaScript testing frameworks to the following list:

JSSpec YUI Test
QUnit Screw-Unit

I chose JSSpec because it uses the behavior driven development terminology that I’ve become accustomed to recently. Screw-Unit also is BDD-based but it didn’t seem to be as straight-forward. QUnit is a newer framework that was originally designed by the jQuery folks as a framework for testing jQuery. I used QUnit on a real project for a client and it got the job done. QUnit seems to be the most talked about of the frameworks just because people find it when their on the jQuery site. (FWIW, QUnit doesn’t have any integration with jQuery that makes it any easier to use than the others.) YUI Test is used with the Yahoo UI framework, so if you’re using the YUI framework, YUI Test may be a good fit for you. People that use YUI Test seem to have a lot of good things to say about it.

JavaScript mocking frameworks

JavaScript mocking frameworks are not as prevalent, partly because it’s really easy in JavaScript to redefine any method in any class to be whatever you want, so you don’t need a mocking framework to redefine methods. But it’s still nice to have a framework that lets you verify that certain methods were called. If you want to look into mocking frameworks, Jack is a good place to start. It’s still in alpha (at the time of this post) but it looks usable to me. It integrates with JSSpec which is another plus.

Next up: writing testable JavaScript.

November 25, 2008by Jon Kruger
JavaScript, TDD, unit testing

Unit Testing JavaScript – Introduction, JavaScript basics

JavaScript and unit testing — two things that you don’t hear about in the same sentence too often.

Why is that? There are all kinds of test frameworks, mocking frameworks, etc. for .NET, and lots of emphasis is placed on the value of testing .NET code. How come you don’t hear the same about JavaScript?

Now I certainly don’t know everyone in the .NET world, but I work with lots of really smart people and talk with lots of others on a daily basis. And yet the following disturbing statements are true:

  • I had never written a line of code to test JavaScript (up until a month ago).
  • I’ve never had anyone I know tell me that they have written a line of code to test JavaScript.
  • Most people I asked didn’t know anything about what JavaScript test frameworks are out there.

How did this happen? Isn’t JavaScript code too? And isn’t it even easier to have bugs in JavaScript since we don’t have the benefit of a compiler?

The same behavior driven development concepts that apply to .NET code can also apply to testing JavaScript. It’s really not that much harder or different than testing .NET code. And since JavaScript is a dynamic language, it some ways it’s even easier.

JavaScript class syntax

Before I dive into testing JavaScript, I suppose I should go over the basics of “classes” in JavaScript.


// JavaScript class
function MyClass()
{
// private variable
var somethingPrivate = null;

// public variable
this.somethingPublic = null;

// private method
function privateMethod()
{
}

// public method
this.publicMethod = function()
{
}
}

// adds a method called doSomething to the definition of MyClass
MyClass.prototype.doSomething = function()
{
}

var instance = new MyClass();

// redefine a public method on an instance of an object
instance.publicMethod = new function()
{
}

Some comments/thoughts:

  • I like to make classes instead of using global variables and functions. Global functions work fine for really simple tasks (e.g. when a button is clicked, open another window), but if you do anything complicated, your code will be much cleaner, well-structured, and easier to test if you create classes just like you would in .NET code.
  • Don’t expose public variables. You wouldn’t expose a private field in a .NET class, so don’t expose a variable in JavaScript either. Instead, create a private variable and then create public methods to get and set the value. Exposing a public variable will open you up to all kinds of odd possibilities (such as someone assigning a function to your variable when you were expecting it to be an integer).

Next up: JavaScript test frameworks.

November 25, 2008by Jon Kruger
Quality, TDD, unit testing

The relative cost of fixing defects

I saw a very interesting chart today…

The Relative Cost of Fixing DefectsSource: http://www.riceconsulting.com/public_pdf/STBC-WM.pdf

If this doesn’t convince you of the value of unit tests, I don’t know what will. The cost to fix bugs in production could be dramatically higher than the cost to fix them in development, which is why having a suite of unit tests that you can run when you make changes is invaluable because it will prevent you from getting into these costly scenarios where you have to fix nasty bugs in production.

November 20, 2008by Jon Kruger
TDD, unit testing

Why Behavior Driven Development is good

On my current project I was introduced to Behavior Driven Development. When I first heard of BDD, I thought, “Is BDD any more than a new way to name your test methods? What’s the big deal?”

Here’s a simple unit test class that I might have written in the past:

[TestClass]
public class EmployeeTests
{
   [TestMethod]
   public void LastNameRequiredTest()
   {
   }

   [TestMethod]
   public void BirthDateCannotBeInFutureTest()
   {
   }

   [TestMethod]
   public void LastWidgetDateTest()
   {
   }
}

When I would write a test like this, the goal was to make sure that the code that I had written was working (I normally wasn’t doing TDD when writing these tests). Now there’s nothing wrong with that, let’s say that my code coverage was good, and things are good.

A BDD-style set of tests would look more like this:

[TestClass]
public class Given_an_Employee
{
   [Setup]
   protected override void Before_each_test()
   {
   }
}

[TestClass]
public class When_validating_an_Employee : Given_an_Employee
{
   [TestMethod]
   public void Then_the_LastName_is_required()
   {
   }

   [TestMethod]
   public void Then_the_BirthDate_cannot_be_in_the_future()
   {
   }
}

[TestClass] 
public void When_an_Employee_makes_a_widget : Given_an_Employee
{
   [TestMethod]
   public void Then_the_LastWidgetDate_should_be_set_to_today()
   {
   }
}

Whoa, what’s with all the classes and underscores and all that! Put your preconceived notions aside and let me explain. :)

First, let me explain why TDD is good. I think that these points illustrate this best.

1. The developer starts writing unit tests around their code using a test framework like JUnit or NUnit.
2. As the body of tests increases the developer begins to enjoy a strongly increased sense of confidence in their work.
3. At some point the developer has the insight (or is shown) that writing the tests before writing the code, helps them to focus on writing only the code that they need.
4. The developer also notices that when they return to some code that they haven’t seen for a while, the tests serve to document how the code works.
5. A point of revelation occurs when the developer realises that writing tests in this way helps them to “discover” the API to their code. TDD has now become a design process.
6. Expertise in TDD begins to dawn at the point where the developer realizes that TDD is about defining behaviour rather than testing.
7. Behaviour is about the interactions between components of the system and so the use of mocking is fundamental to advanced TDD.

First you begin to write tests because you want to make sure your code works. But as you get into TDD, you realize that TDD really can be more about design than testing. If you can define acceptance criteria up front, you will be forced to think about what your code is going to need to do and design it accordingly.

BDD takes the next step. First your tests were just testing your code, then they were designing your code, and now they are describing what the code is doing. This is where the weird class and method names come in. Let me write out those tests again, translating it from C# into real world language.

Given an Employee, when validating an Employee, then the LastName is required and the BirthDate cannot be in the future. When an Employee makes a widget, then the LastWidgetDate should be set to today.

Here is where the light bulbs clicked on for me. My tests aren’t just testing my code, they are telling a story. Imagine if you were taking this code over from me and you wondered, “What is important about an Employee?” You could look at my tests and you would know exactly what is important about an Employee. You would immediately know what all of the acceptance criteria were for working with an Employee. So now you can go and change my code and have a good sense of what I was trying to make my code do!

I think by now you can see how TDD and BDD will go hand in hand. It should go something like this:

1) You get your requirements. In the requirements are a list of business rules or acceptance criteria.
2) You write your tests (or at least the classes and methods). Now you’ve translated those business rules and acceptance criteria into tests.
3) You write your code. You already have a suite of tests that you can run to know if you succeeded!

Of course, you may go back after the fact and write even more unit tests that focus on testing more granular parts of your implementation than just the overall business rules.

If you don’t like all the funky underscores that I put in the class and method names, then that’s fine. That’s just how we did it on our project (it’s all this guy’s fault). I like them because it really does help the names to read more like normal human sentences, which is much more valuable that I had originally thought.

TDD and BDD are a definite paradigm shift for a lot of people. But once you get used to it, you will never go back.

July 25, 2008by Jon Kruger
unit testing

Why mocking is good

As I said in my last post, unit testing is hard, and it’s something that can be hard to learn.

I’ve been fortunate on my current project to be working with guys who introduced me to mocking frameworks (in our case, Rhino Mocks). I had heard of Rhino Mocks before, but I never really looked into it. I thought, if I’m not testing against a database, how is that a valid test of what will actually happen in the application?

I can think of a very simple example that I think will show the value of mocks. Let’s say that it’s your first day on a brand new, green field project where you’re writing a system that deals with claims. You create a simple Claim object and you want to write a unit test to make sure that your validation code checks to see if the ClaimNumber property has a value. Your tests (not using mocks) might look like this:


[TestMethod]
public void SaveClaimPositiveTest()
{
    ClaimLogic logic = new ClaimLogic();
    Claim c = new Claim();
    c.ClaimNumber = "12345";
    logic.Save(c);
    
    Claim loadedClaim = logic.Get(c.Id);
    Assert.IsTrue(loadedClaim != null);
    Assert.IsTrue(loadedClaim.ClaimNumber == c.ClaimNumber);
}

[TestMethod]
public void SaveClaimNegativeTest()
{
    ClaimLogic logic = new ClaimLogic();
    Claim c = new Claim();
    c.ClaimNumber = null;
    try
    {
        logic.Save(c);
        Assert.Fail("Expected exception was not thrown.");
    }
    catch (MyValidationException ex)
    {
        Assert.IsTrue(ex.Property == "ClaimNumber", "Validation for ClaimNumber did not occur.");
    }
}

These are fairly simple tests that I’ve written many times in the past. What is wrong with these tests? Nothing right now. But that is all going to change.

What happens when someone adds a new property to the Claim object and adds a validation rule that says that the new field is required? My tests break. Now I have to go back and add more code to the beginning to create a valid Claim object that I can save.

What happens when someone adds some more logic to the Save method that kicks off some complicated workflow? Now my simple tests are getting a lot more complicated, and I’m going to have to deal with all kinds of side effects and errors just to test a really simple validation rule.

I’m also cluttering up the database with test data. So I need to write some cleanup code to delete the Claim object I inserted. Later on someone will add some foreign key relationships to the table, and I’ll have to come back add more code to clean up those related objects too. Someone might add some triggers too, and I’ll have to account for that. Those triggers might make it hard (or impossible) to clean up this test data.

These tests are also going to be pretty slow because they have to interact with the database. Yes, it’s a simple test, but what happens when I have 2000 of these tests? It’s going to take a long time to run them all!

All I wanted to do was verify whether my validation code was written correctly! Why do I have to actually try and save the object to the database to do this?

What happened on past projects where I wrote these kinds of tests was this: the unit tests take 5 minutes to run, so no one runs them when they check in. Eventually someone does something that breaks the test (usually harmless stuff like adding validation rules), but no one takes time to fix it. Before you know it, half of the unit tests are broken, but it’s getting close to your release date, so you don’t have time to fix them. Then, sometime after the release, someone will spend a week updating the tests so that they all work again.

This is where mocking comes in.

I’m not going to go in depth into mocking because there are plenty of people who have written in depth posts on how to do this. But the basic idea is that I am going to “mock out” external dependencies by telling the mocking framework what those external dependencies are going to do and return. In this case, I’ll mock out the code that would actually do the actual saving to the database because all I’m testing is the validation.

Yesterday I wrote a unit test for a method that returned a list of users in the application who were in certain security roles, and the list of users returned depended on the current user’s security role (if you’re in certain roles, I don’t want the list of users to include people in certain other roles). The list of users is stored in a database table, and the security roles are stored in Active Directory groups.

In this case, it would be pretty much impossible to test this without mocks. Think about what I have to do:

1) Test that the list of users returned will be correct depending on the user’s security role
2) Test that users in each security role will be returned at the proper time

The problem is that I don’t know what users are in the database and I don’t know what users are in each role in Active Directory. I don’t have a way to write code to insert users into Active Directory groups. Even if I could do that, I would then have to write code to insert a bunch of test users into the database (or even worse, write a test that would expect certain actual users to be in the database). Then I’d have to clean everything up when the test finished.

I don’t need a database or Active Directory to test my logic that would return the correct list of users. I can mock out the database by just using a list of User objects instead, and I can mock out Active Directory by just telling the mock framework what users to return for each role.

I wrote 16 tests around this section of code and they all run in about 5 seconds! I don’t have to insert any test code anywhere, I don’t have to clean it up, I don’t have to worry about people adding new validation rules, and I don’t have to worry about people adding other external code that will break my test (that wouldn’t otherwise have an effect on the code I was testing).

I will say this — mocking is not easy. It really is an art. It is not something that I learned overnight. Even though I understood the concept, I didn’t really get good at it until recently. But now that I understand it, I write much better unit tests, I write more unit tests, and I can do it pretty quickly. Test Driven Development (writing tests first) is also an art, and it’s something that I’m just starting to get into.

When I first started programming, I didn’t care about unit tests because I just wanted my programs to work. Sadly, in my four years of college, no one even mentioned a unit test. I didn’t even know what one was when I started my first job! Trying to reprogram yourself to work in a TDD way is hard because you’re breaking old habits that you’ve had since the day you first started programming.

Mocking and TDD will revolutionize the way that you write unit tests, but like I said, it can feel like you’re learning a foreign language. If you have someone that you know that is good at this stuff, beg them, plead with them, take them out to lunch, or do whatever else you have to do to get them to sit with you and pair program for a day or two and learn from them. You won’t want to go back!

Also, if you don’t have a continuous integration build that runs all of your unit tests every time you check in, get one. We’re using TFS 2008 and I created a CI build using TFS in less than 5 minutes using the TFS build definition wizard. Then make sure that you fix broken tests as soon as they break!

Happy mocking!

June 30, 2008by Jon Kruger
unit testing

Why write unit tests?

Unit testing is always a hot topic on every project. Usually in the form of, “Why should I write unit tests?” or “Do we have enough code coverage?”

This is a not an implication of anyone that I’ve worked with. Lots of people can write code, but writing unit tests is definitely a skill that takes awhile to learn. So if I come across people who don’t know how to write unit tests, I don’t get too bent out of shape because it’s takes time to get good at it.

Let’s go back to the original question — Why should I write unit tests?

How many of you have ever had to debug, maintain, or replace legacy software? I’m in the process of doing that right now. Now not all of the code that we’re replacing is bad, and some of it is still useful. The problem is that we can’t change any of it because we don’t know what our changes will break. That’s because we don’t know what the intentions of the original developers were, and we have no way of verifying whether our changes were successful (what is the acceptance criteria?). As a result, we end up having to rewrite code that otherwise might be perfectly good code because we have to make some minor change to it. Since we can’t modify it safely, we have to rewrite the code and everything that depends on it.

Legacy software is the obvious example. Let’s think about your own code.

On my project, there are 15 developers. 15 developers is a lot of people, and we’re stepping on each other toes all the time. Sure, I can write code and just step through it in the debugger and verify that it’s working, but what happens when someone writes code tomorrow that will break my code? How will they know that they broke it, and how will I know that they broke it? Heck, I break my own code all the time, how is someone else supposed to not break it?

Unit tests take time to write, but they will usually save time in the end. In many cases, you’ll find bugs in your code before it gets to QA, so QA doesn’t have to spend time testing it, writing up the bug, retesting it. Then when you have to change your code later on (or if someone else has to change it), the unit tests can tell you if you broke something, saving more QA time. Unit testing also makes you really think about your code and all of the possible edge cases. Sometimes it’s really hard (or impossible) to recreate an edge case when testing through the app and it’s much easier to recreate it in a unit test. Remember, if your code allows for a certain edge case to happen, you should write unit tests around it even if you can’t recreate that edge case when running your application. Just because you can’t do it now doesn’t mean that someone won’t try and use your code to do it later.

How many projects have you been on where you’ve had to make a big change just before the app when into production or while the app was in production? I’ve had to do this many times. Unit tests probably won’t catch everything that can go wrong in this situation, but it sure is nice being able to run 2000 tests to see what I might have broke.

The important thing to remember is that we should be writing software that is going to last as long as possible. This is really hard to keep in mind when you have a deadline looming, or you’re trying to figure out a particular problem.

Nothing lasts forever – business requirements change, programming languages change, development teams change, everything changes. But it’s my responsibility to write software that will last as long as possible. I want my software to get replaced because changes in technology or the business environment allowed them to write something much better than what I wrote. I don’t want them to replace my software because they couldn’t maintain it.

June 24, 2008by Jon Kruger
Page 3 of 3«123

About Me

I am a technical leader and software developer in Columbus, OH. Find out more here...

I am a technical leader and software developer in Columbus, OH, currently working as a Senior Engineering Manager at Upstart. Find out more here...