Here are the slides and code from the Unit Testing JavaScript talk. Or if you one of those people you can get it all from the Subversion repository at http://minisudoku.googlecode.com/svn/trunk/.

(UPDATE 2/25/09: these links point to the latest versions of the slides and the code, which now include examples using both JSSpec and QUnit.)

So far I’ve talked about JavaScript basics, JavaScript test frameworks, and writing testable code. Now for the real fun… let’s do some test driven development!

I know that TDD seems somewhat backwards to a lot of people, and it’s hard to undo the years and years of not writing tests. I learned from watching people doing it and learning the thought process and reprogramming my head so that I could think in a new way. So hopefully this will help you understand the thought process.

Step 1 – Get your requirements

So you’re about to start on a new feature or fix a bug. Either way, you will get requirements in some form.

In this case, we want to create a greatly slimmed down version of Sudoku. Here are the rules:

• The puzzle will have only 4 boxes in a 2×2 grid.
• You can enter numbers 1 through 4 in a box by selecting a box and pressing the number keys on the keyboard
• Selecting a box and hitting the Delete key should remove the value in the box
• You can change the numbers or remove the numbers at any time
• Give the user a button to click that will check to see if the puzzle is valid
• Give the user a button that will give the user a new puzzle. A new puzzle will have 2 of the numbers already filled out.
• A puzzle is valid if:
  • Each box has a number 1 through 4
  • Each box has a different number

OK, so what do you do now? Most people would start hammering out some JavaScript classes. But we’re not going to do that, because we’re not done with the design phase yet.

What do I mean by that? Well, one of the big reasons that we write tests first is to help us design our code. Sure, you could write tests after you write the actual code. But if you do it that way, you will spend lots of time refactoring code (that you don’t have tests around yet) in order to make it testable. That’s wasted time, so we’ll do our design up front by outlining our tests, writing the tests, and then writing the code.

Step 2 – Refine the requirements

Let’s look at the first bullet point in our requirements:
The puzzle will have only 4 boxes in a 2×2 grid.

Business requirements aren’t always written out as individual acceptance criteria, so let’s split this line into individual requirements and write them out.

• The puzzle should have 4 boxes
• The boxes should be in a 2×2 grid

Step 3 – Write out the test methods

Now that we have that, let’s translate that into tests:


describe('When creating a new Puzzle',
{
'The Puzzle should have 4 Boxes': function()
{
}

'The boxes should be in a 2x2 grid': function()
{
}
});


Step 4 – Stub out implementation classes

Now we have two tests. These tests will help us design our code. Let’s see what we can learn from these tests:

• We need a Puzzle object
• We need a Box object
• The Puzzle should have a collection of Boxes
• We need a way to ask the Puzzle for a box at a given coordinate in the grid

Let’s write these classes and methods, but we won’t implement the methods yet.


function Puzzle()
{
var boxes = new Array();

this.getRowIndex = function()
{
return rowIndex;
}
}


Step 5 – Write test code

Now that we have our objects, we can write the code inside our test methods (I’m using JSSpec as my testing framework):


describe('When creating a new Puzzle',
{
before_each: function()
{
puzzle = new Puzzle();
},

'The boxes should be in a 2x2 grid': function()
{
for (var columnIndex = 0; columnIndex < 2; columnIndex++) { for (var rowIndex = 0; rowIndex < 2; rowIndex++) { var box = puzzle.getBox(columnIndex, rowIndex); value_of(box.getColumnIndex()).should_be(columnIndex); value_of(box.getRowIndex()).should_be(rowIndex); } } } });

We've written our tests. Now let's run them and watch them fail. The reason we do this is because if these tests pass now, then they're probably not written correctly and we don't want false positives.

Yep, they're failing, just as I expected.

Step 6 - Write implementation code

Now, yes now, you finally can write your implementation code. I know that some of you have been chomping at the bit to get to this point. So here we go.


function Puzzle()
{
var boxes = new Array();

for (var columnIndex = 0; columnIndex < 2; columnIndex++) { for (var rowIndex = 0; rowIndex < 2; rowIndex++) { var box = new Box(columnIndex, rowIndex); if (boxes[columnIndex] == undefined) boxes[columnIndex] = new Array(); boxes[columnIndex][rowIndex] = box; } } this.getAllBoxes = function() { var list = new Array(); for (var column = 0; column < boxes.length; column++) { for (var row = 0; row < boxes[column].length; row++) list.push(boxes[column][row]); } return list; } this.getBox = function(columnIndex, rowIndex) { return boxes[columnIndex][rowIndex]; } } function Box(columnIndex, rowIndex) { this.getColumnIndex = function() { return columnIndex; } this.getRowIndex = function() { return rowIndex; } }

Let's run the tests and see if the code works.

Sweet goodness. But I've only implemented one of the requirements. No problem, I'll just go back to step 1 again and go through the process with the next requirement and repeat until I'm done with all of the requirements.

Let's take a look at what we have now.

• We have code that we know is working correctly
• We have tests to prove that the code is working correctly, both now and from now on
• We have tests that describe what the code is supposed to do. Our tests are documenting our code!
• We have well designed classes
• Our classes do only what they're supposed to do -- we didn't waste time writing things that we don't need
• We have peace of mind!!

Let me talk about this peace of mind. This is something that I can describe to you but you can't understand it until you experience it. I am 99% sure that the code that I wrote is working now and will continue to work in the future. I can refactor my code or make changes and I know that it will be working correctly when I'm done. I can guarantee you that the only way that my code has bugs is if there is some requirement that I didn't know about. Can you say that about your code?

What happens when it's a day or two before you put something into production and they find a nasty bug that absolutely has to be fixed before you go live? (Certainly not the ideal situation, but it happens all the time.) Wouldn't you love to have a suite of tests to run so that you can be less likely to break something else when you fix the bug? (And remember, bugs are much more costly and difficult to fix when they're in production.)

Have you ever been in that endless cycle of fixing a bug only to break something else, and then when you fix that bug you break something else, and on an on?

This is why I write tests. And this is why I will never go back.

How to get started with TDD

People ask me how to get started doing TDD. The best way is to just do it. The next time you have a simple feature or a simple bug to fix, write your tests first. It takes awhile to get into the mindset, but once you do it a few times it starts to make a lot more sense. If you know someone else who is good at TDD, testing, mocking, etc., pair program with them for a day and watch how they work. You'll need to get better at separation of concerns, programming to interfaces, and the like. Maybe that will be my next post. :)

Completed sample project

I went through the entire exercise with the MiniSudoku project. You can download the finished product here. (UPDATE 2/25/09: this link contains examples using both JSSpec and QUnit.)

Good luck!

Last time we talked about JavaScript test frameworks. But a test framework is no good if you don’t write testable code.

Writing testable JavaScript

As is the case with .NET code, one of the most important parts of testing is writing code that you can actually test. Let’s look at some untestable JavaScript code and what we can do to make it testable.


// Untestable JavaScript :(
function MyClass
{
this.showMessage = function(message)
{
$("#messageTextBox").text(message);
}

this.anotherShowMessage = function(message)
{
document.getElementById("anotherMessageTextBox").innerText = message;
}
}


This code is untestable because it directly references UI elements in a page by name. When you run unit tests using pretty much any JavaScript testing framework, you create a separate page to run your tests from. That means that your UI elements will not be on that page, so any code that references these elements by name will not be testable.

Here’s how you would make this testable:


// Testable JavaScript!
function MyClass()
{
var element;

value_of(myClass.getElement().innerText).should_be('I love testing!');
}
});


Notice how MyClass still has the notion of a UI element, but now the UI element is injected by whoever is using the class. And since JavaScript is a dynamic language, this is easy to test because we don’t need an actual UI element in our unit test, we just need an object that has an “innerText” variable on it. Creating said object is as easy as doing this:

var myFakeUIElement = new function() { this.innerText = null; }

This is what I can do in my test. I tell MyClass that its element is my fake UI element, I call showMessage(), and then I verify that the innerText variable of my fake UI element got set.

Another example:


// Untestable JavaScript :(
function MyClass()
{
this.showMessage = function(message)
{
alert(message);
}
}


I think this one is pretty obvious. If you try and test this method, you’re going to have an alert box popping up on the screen in the middle of your tests. Not going to work. Try this instead:


// Testable JavaScript
function MyClass()
{
this.showMessage = function(message)
{
this.showAlertBox(message);
}

value_of(alertMessage).should_be('I love testing!');
}
});


Here I separated my concerns again — I created a separate method that only shows the alert box. I can “mock” this method by simply redefining it to whatever I want (in this case I’m just storing what was passed into it). Now I can test that when I call MyClass.showMessage(), I can verify that it called the method that would show an alert box.

Time for more sadness:


// Untestable JavaScript :(
function MyClass()
{
this.result = null;
this.loadCustomerData = function(customerId)
{
$.ajax({
type: "POST",
url: "CustomerService.asmx/LoadCustomer",
data: "customerId=" + customerId,
success: function(msg)
{
this.result = msg.d; // return value from web service
}
});
}
}


This is an asynchronous Ajax call using jQuery. The problem here is that because this is actually making an asychronous Ajax call, the method will return and our test code will run before the callback executes.

There are two ways to solve this:

Solution 1:

// Testable JavaScript
function MyClass()
{
this.result = null;
this.loadCustomerData = function(customerId)
{
$.ajax({
async: false,
type: "POST",
url: "CustomerService.asmx/LoadCustomer",
data: "customerId=" + customerId,
success: function(msg)
{
this.result = msg.d; // return value from web service
}
});
}
}

describe('When loading a Customer',
{
'The customer name should be returned': function()
{
var alertMessage;
var myClass = new MyClass();
myClass.loadCustomerData(1);
value_of(myClass.result).should_be('Jon');
}
});


What changed here? I added “async: false” to the $.ajax() call. Now jQuery will run this method synchronously and I will be able to test it.

Caveat: This violates the basic rules of “what is a unit test” because an external dependency is involved (in this case, my .asmx web service). You can decide for yourself whether this is bad or not. In my case, the test still runs lightning fast because all it does it return some random numbers and it never hits a database, so it doesn’t bother me too much. But if you have a more complicated web service that you’re running and it takes a long time to return data, you may not want to do it this way. Here’s another way to fix this:

Solution 2:

function MyClass()
{
this.result = null;
this.loadCustomerData = function(customerId)
{
var myClass = this;
this.makeAjaxCall(
"POST",
"CustomerService.asmx/LoadCustomer",
"customerId=" + customerId,
function(ajaxResult) { myClass.result = ajaxResult; });
}

ajaxSuccessCallback('somevalue');
value_of(myClass.result).should_be('somevalue');
}
});


Slightly more complicated doing it this way, but if the situation warrants it, it’s worth it.

Next stop: we’ll do some test driven development and see this stuff in action!

In my last post I gave you some basic information on JavaScript that will help you get started on the road to testing your JavaScript. Now for the JavaScript test frameworks.

JavaScript test frameworks

I did some research and there are quite a few JavaScript test frameworks out there:

So apparently there are people that are trying to solve this problem, but how do you choose between all of these frameworks?

When I want to determine the relevance of something, I go straight to search.twitter.com. I figure that if no one is talking about a JavaScript framework on Twitter or if something is getting a lot of negative comments, then it’s probably not worth looking into.

Using my completely unscientific Measure of Relevance I was able to narrow down my list of JavaScript testing frameworks to the following list:

I chose JSSpec because it uses the behavior driven development terminology that I’ve become accustomed to recently. Screw-Unit also is BDD-based but it didn’t seem to be as straight-forward. QUnit is a newer framework that was originally designed by the jQuery folks as a framework for testing jQuery. I used QUnit on a real project for a client and it got the job done. QUnit seems to be the most talked about of the frameworks just because people find it when their on the jQuery site. (FWIW, QUnit doesn’t have any integration with jQuery that makes it any easier to use than the others.) YUI Test is used with the Yahoo UI framework, so if you’re using the YUI framework, YUI Test may be a good fit for you. People that use YUI Test seem to have a lot of good things to say about it.

JavaScript mocking frameworks

JavaScript mocking frameworks are not as prevalent, partly because it’s really easy in JavaScript to redefine any method in any class to be whatever you want, so you don’t need a mocking framework to redefine methods. But it’s still nice to have a framework that lets you verify that certain methods were called. If you want to look into mocking frameworks, Jack is a good place to start. It’s still in alpha (at the time of this post) but it looks usable to me. It integrates with JSSpec which is another plus.

Next up: writing testable JavaScript.

JavaScript and unit testing — two things that you don’t hear about in the same sentence too often.

Why is that? There are all kinds of test frameworks, mocking frameworks, etc. for .NET, and lots of emphasis is placed on the value of testing .NET code. How come you don’t hear the same about JavaScript?

Now I certainly don’t know everyone in the .NET world, but I work with lots of really smart people and talk with lots of others on a daily basis. And yet the following disturbing statements are true:

• I had never written a line of code to test JavaScript (up until a month ago).
• I’ve never had anyone I know tell me that they have written a line of code to test JavaScript.
• Most people I asked didn’t know anything about what JavaScript test frameworks are out there.

How did this happen? Isn’t JavaScript code too? And isn’t it even easier to have bugs in JavaScript since we don’t have the benefit of a compiler?

The same behavior driven development concepts that apply to .NET code can also apply to testing JavaScript. It’s really not that much harder or different than testing .NET code. And since JavaScript is a dynamic language, it some ways it’s even easier.

JavaScript class syntax

Before I dive into testing JavaScript, I suppose I should go over the basics of “classes” in JavaScript.


// JavaScript class
function MyClass()
{
// private variable
var somethingPrivate = null;

Some comments/thoughts:

• I like to make classes instead of using global variables and functions. Global functions work fine for really simple tasks (e.g. when a button is clicked, open another window), but if you do anything complicated, your code will be much cleaner, well-structured, and easier to test if you create classes just like you would in .NET code.
• Don’t expose public variables. You wouldn’t expose a private field in a .NET class, so don’t expose a variable in JavaScript either. Instead, create a private variable and then create public methods to get and set the value. Exposing a public variable will open you up to all kinds of odd possibilities (such as someone assigning a function to your variable when you were expecting it to be an integer).

Next up: JavaScript test frameworks.

I saw a very interesting chart today…

Source: http://www.riceconsulting.com/public_pdf/STBC-WM.pdf

If this doesn’t convince you of the value of unit tests, I don’t know what will. The cost to fix bugs in production could be dramatically higher than the cost to fix them in development, which is why having a suite of unit tests that you can run when you make changes is invaluable because it will prevent you from getting into these costly scenarios where you have to fix nasty bugs in production.

Estimation of features is a critical part of the software development process, especially if you are using an Agile methodology. The ability to accurately estimate features is an often overlooked skill that every developer should have.

As a project engineer, I’m responsible for making sure that we set realistic project timelines and that we actually carry those out. Managing expectations is one of my main responsibilities. This is where Agile is so nice — as long as you do it right.

When I start on a new project, usually one of the first things that happens is meeting with the client or whoever is sponsoring the project. In these initial meetings, at some point I will probably be asked one of two things: 1) how long will it take to accomplish X? 2) how much can we get done before X (some date)?

Let’s think about what is happening here. Your project sponsor is asking you to define what is realistic! This is great! You have one chance to set realistic expectations, and that time is now. Of course, you have to be fair — if you try to sandbag and say that it will take 4 days to add a button on a page, you will probably get called out. But even worse is if you underestimate things, because you will probably be held to your estimates. Once you give the sponsor the project timeline, they will take it to their bosses and spread the word around. So if you don’t get your project done on time or if it’s over budget, then the sponsor is in hot water. Not good.

If you aren’t asked one these two questions (i.e. they say, “We need to have X done by Y.”), that can still be OK. But if what they’re asking for isn’t realistic, now is the time to do something about it. No one likes to be told that what they want to do can’t be done or isn’t realistic. But I guarantee that they will be happier knowing about that up front than when you miss your deadlines.

Things to consider when estimating

When we estimate on our projects, we break functionality into features. Our criteria for features is that they should (usually) be something that can be tested and usually takes anywhere from 1/2 day to 10 days. There are lots of ways to do Agile and people do Agile in different ways, but most people break things up into features in some way.

The biggest problem I see with estimates is that people frequently underestimate things, and that’s usually because they don’t think of everything that goes into developing a feature. When I estimate, here are all of the things that I take into account:

• Reviewing the specs
• Asking questions about the specs and chasing down answers
• Doing the technical design and reviewing it with others on the team
• Writing the code
• Writing the unit tests
• Manually testing the code
• Fixing however many bugs you expect to have
• Working with external servers, web services, and other things that might slow you down
• Extra time if there are lots of uncertainties

In some cases, there might be more things to take into account.

One line that we always say about estimates is that “all estimates are wrong.” This statement is true on a feature-by-feature basis, but not for the project as a whole. This is because your project sponsor is going to hold you to your estimates. Whether that’s always fair or not is debatable, but that’s the way it is. You may not have all of the answers when you estimate a feature, and that decreases the chance that your estimate will be correct. So we need to try and have as many questions answered as possible before doing the estimating.

Sometimes you’ll start work on a project and realize that you’ve underestimated everything. This is not a good thing, but it happens. When this happens, you should re-estimate the features again so that your estimates can be accurate.

You’re also going to have the not-so-fun situation where you have to let your project sponsor know that it’s going to take longer to finish than you originally thought. Obviously this is not going to be a fun conversation, but the time it takes to develop a feature is the time it takes to develop a feature. You can change that reality. So the sooner you can adjust the expectations, the better chance you have of refocusing the project and righting the ship.

Estimating is a great tool for tracking progress and managing expectations, and it’s a reason that I’m such a big fan of Agile. It’s also a skill that every developer needs to master.

Recently I lost several team members from my team. As a result, suddenly there is all this top-of-the-line hardware lying around. Of course, I can’t let good hardware sit there and rot! So now I have the developer’s dream environment.

You are looking at my quad-core, 64-bit, dual-boot system with 4 20″ flat panels and 8 GB of RAM. Synergy let’s me use one mouse and one keyboard to control both machines. Now that’s hot!!

On my current project I was introduced to Behavior Driven Development. When I first heard of BDD, I thought, “Is BDD any more than a new way to name your test methods? What’s the big deal?”

Here’s a simple unit test class that I might have written in the past:

[TestClass]
public class EmployeeTests
{
   [TestMethod]
   public void LastNameRequiredTest()
   {
   }

   [TestMethod]
   public void BirthDateCannotBeInFutureTest()
   {
   }

   [TestMethod]
   public void LastWidgetDateTest()
   {
   }
}

When I would write a test like this, the goal was to make sure that the code that I had written was working (I normally wasn’t doing TDD when writing these tests). Now there’s nothing wrong with that, let’s say that my code coverage was good, and things are good.

A BDD-style set of tests would look more like this:

[TestClass]
public class Given_an_Employee
{
   [Setup]
   protected override void Before_each_test()
   {
   }
}

[TestClass]
public class When_validating_an_Employee : Given_an_Employee
{
   [TestMethod]
   public void Then_the_LastName_is_required()
   {
   }

   [TestMethod]
   public void Then_the_BirthDate_cannot_be_in_the_future()
   {
   }
}

[TestClass] 
public void When_an_Employee_makes_a_widget : Given_an_Employee
{
   [TestMethod]
   public void Then_the_LastWidgetDate_should_be_set_to_today()
   {
   }
}

Whoa, what’s with all the classes and underscores and all that! Put your preconceived notions aside and let me explain. :)

First, let me explain why TDD is good. I think that these points illustrate this best.

1. The developer starts writing unit tests around their code using a test framework like JUnit or NUnit.
2. As the body of tests increases the developer begins to enjoy a strongly increased sense of confidence in their work.
3. At some point the developer has the insight (or is shown) that writing the tests before writing the code, helps them to focus on writing only the code that they need.
4. The developer also notices that when they return to some code that they haven’t seen for a while, the tests serve to document how the code works.
5. A point of revelation occurs when the developer realises that writing tests in this way helps them to “discover” the API to their code. TDD has now become a design process.
6. Expertise in TDD begins to dawn at the point where the developer realizes that TDD is about defining behaviour rather than testing.
7. Behaviour is about the interactions between components of the system and so the use of mocking is fundamental to advanced TDD.

First you begin to write tests because you want to make sure your code works. But as you get into TDD, you realize that TDD really can be more about design than testing. If you can define acceptance criteria up front, you will be forced to think about what your code is going to need to do and design it accordingly.

BDD takes the next step. First your tests were just testing your code, then they were designing your code, and now they are describing what the code is doing. This is where the weird class and method names come in. Let me write out those tests again, translating it from C# into real world language.

Given an Employee, when validating an Employee, then the LastName is required and the BirthDate cannot be in the future. When an Employee makes a widget, then the LastWidgetDate should be set to today.

Here is where the light bulbs clicked on for me. My tests aren’t just testing my code, they are telling a story. Imagine if you were taking this code over from me and you wondered, “What is important about an Employee?” You could look at my tests and you would know exactly what is important about an Employee. You would immediately know what all of the acceptance criteria were for working with an Employee. So now you can go and change my code and have a good sense of what I was trying to make my code do!

I think by now you can see how TDD and BDD will go hand in hand. It should go something like this:

1) You get your requirements. In the requirements are a list of business rules or acceptance criteria.
2) You write your tests (or at least the classes and methods). Now you’ve translated those business rules and acceptance criteria into tests.
3) You write your code. You already have a suite of tests that you can run to know if you succeeded!

Of course, you may go back after the fact and write even more unit tests that focus on testing more granular parts of your implementation than just the overall business rules.

If you don’t like all the funky underscores that I put in the class and method names, then that’s fine. That’s just how we did it on our project (it’s all this guy’s fault). I like them because it really does help the names to read more like normal human sentences, which is much more valuable that I had originally thought.

TDD and BDD are a definite paradigm shift for a lot of people. But once you get used to it, you will never go back.

I’ve had lots of positive responses to my last post from other people that were involved in the original discussion. I just wanted to say that I was not the least bit offended by anything that was said by anyone in the original meeting. I don’t think that I could ever be upset at anyone for disagreeing with me on technical issues.

One thing that I really like about software development and architecture in general is that every project is a new challenge with new requirements that will affect how you architect a solution. So we need to be able to evaluate each situation and then evaluate what architecture fits the situation best.

This is why having discussion and disagreements is good. None of us know it all, and we can all learn from each other. My goal is not just to learn what ORM to use, or how to use WCF, or anything like that. I want to learn how to evaluate each situation so that I can design the system appropriately and use the tools that will help me get the job done.