I’ve always said that software development is a series of translations — that we need to incrementally translate business processes to business requirements to acceptance criteria to test plans to tests to code to working software. This is why we write acceptance criteria in the Given/When/Then syntax — it’s a common language that developers, BAs, testers, and business people can use to make sure that we’re all speaking the same language. These kinds of requirements define what the system is supposed to do for the business.

All this is good, but there still are problems. Let’s say that your BA writes some requirements and you implement a feature. Several months later, you get a new requirement to change that code, or someone asks you what the code does for a certain feature. How do you find the code for that feature?

Or maybe you have some common validations that happen many times throughout the application. Are your BAs copying and pasting the rules for that validation over and over in the requirements documents?

What seems to be missing here is another layer of translation. We have too big of a jump from our typical business requirements to our code, and no way to map the two to each other. The acceptance criteria tell us what the code should do, but nothing tells us how to do it.

On my current project, they’ve come up with an ingenious way to solve this problem. (I have to preface all of this by saying that none of this was my idea and I can take zero credit for it. They were doing it long before I joined the team.)

Our system does transaction processing with incredibly complicated business rules. All operations are written as command classes and workflow classes (which are a series of commands).

Our requirements are structured the exact same way. In the requirements we have commands and workflows that are named the same as the classes in the code.

Example

Let’s say I’m building an order processing system. Here’s how we might break it all down.

High-level business requirements

When an order is processed, the order total is sum of the cost of the products + tax + shipping. Save the order information and the total price to the database.

Tax depends on where the order is being shipped:
Ohio – 7%
Michigan – 6.5%
Any other state – 0%

Shipping:
If sum of the cost of the products >= 25, shipping is free, otherwise $5.

System requirements

Workflow: Process Order
1. CalculatePriceOfProducts
2. CalculateTax
3. CalculateShipping
4. SaveProcessedOrder

Command: CalculatePriceOfProducts
Total Price = sum of price of each product on the order

Command: CalculateTax
Tax is calculated based on state the order is being shipped to. Can ship to US states only (including DC).
Ohio – 7%
Michigan – 6.5%
Any other state – 0%

Command: CalculateShipping
If total price of products in the order >= 25, shipping = $0
Otherwise, shipping = $5.

Command: SaveProcessedOrder
Save the following information to the database:
– Products in the order (save current price in case the price of a product changes after the order has been processed)
– Tax amount
– Shipping amount

Acceptance tests

We’ll write unit tests for each command. There are 153 different combinations of shipping (>25, =25, <25) and tax (51 states), so our unit tests will cover all of the permutations. We’ll write acceptance tests to cover some of the most likely scenarios.

Scenario: multiple products with total < 25, shipped to Ohio 
    Given an order with the following products:
        | Product  | Price | 
        | Mouse    | 4.00  |
        | Keyboard | 20.99 |
    And the order is being shipped to Ohio
    When I process the order
    Then the tax should be $1.75
    And the shipping should be $5.00
    And the total price should be $31.74

Scenario: one product with total = 25, shipped to Michigan 
    Given an order with the following products:
        | Product | Price | 
        | Mouse   | 25.00 |
    And the order is being shipped to Michigan
    When I process the order
    Then the tax should be $1.63
    And the shipping should be $0.00
    And the total price should be $26.63

Scenario: one product with total > 25, shipped to Indiana 
    Given an order with the following products:
        | Product | Price | 
        | Mouse   | 25.01 |
    And the order is being shipped to Indiana
    When I process the order
    Then the tax should be $0.00
    And the shipping should be $0.00
    And the total price should be $25.01

Why this is awesome

This has made everything so much easier for everyone. Developers and system analysts can easily communicate because we speak the same language. When we get new requirements that change existing code, the systems analysts even specify which commands and workflows we need to change. Now I know exactly where I need to go to change code without having to do any research!

Duplication is bad in code and it’s bad in requirements. When a new workflow needs to be created, they can reference existing commands without having to copy and paste requirements. The commands and workflows are in a big Word document with links so that I can easily navigate around the document.

Because all of the commands and workflows are outlined before we write the code, most of our design work is done for us. I know what classes I need to create. I know the specific things that each command needs to do, and I can easily unit test them. We have some older code in our application from before they switched to writing requirements in the command pattern. These classes tend to be too big and they’re much harder to decipher, partially because I can’t easily figure out how they map to the requirements. Every time we encounter one of these, we want to refactor it to match the requirements.

Estimation has become a lot easier. I can look at each command in a workflow and get a sense of how hard each one is (or which commands are existing commands that I can reuse), and I already know how I’m going to design my code for the most part.

Keeping requirements up to date is a challenge on any project, but this has helped us do a pretty good job of it. But when you have well structured requirements that don’t have stuff copied and pasted all over the place and you can easily compare a class and a section of the requirements with the same name, updating requirements becomes a lot easier.

The command pattern is a good fit for our application, but your app might not lend itself to the command pattern as well as ours. That’s fine, see if you can come up with another pattern if you want. But sit down as early as possible and see if you can come up with a way to structure your requirements and code using the same patterns.

As you can see in the example, we still have acceptance criteria in the Given/When/Then format. The acceptance criteria help define what the system is supposed to do and how we’re going to test it. The business requirements that we write in the command pattern define how the system is going to accomplish those goals. This method has filled that translation gap between business-focused requirements and code and has had a huge impact on our project.

Ah, requirements. Gathering requirements is really hard. I’m not a BA, but like most developers, we’ve had to gather requirements at some point. You have to meet with users who might not have enough time for you or might not be able to make up their mind. Then you have to take their ideas and understand the business problem enough so that you can write up requirements. You have to try and find the balance between giving the users what they’re asking for and giving the users what they really need but don’t know it yet. It’s a fine line for sure.

I’m not going to sit here and bash BAs for lousy requirements. I’ve been the one who wrote the requirements only to have a developer mock the requirements (they probably didn’t know that I wrote them or that I was listening, but that’s neither here nor there). I tried really hard and I thought I did a good job. But nevertheless, there were things that I didn’t really think of that the other developers thought of.

Since we know that requirements gathering is hard, what can we do to make sure that we don’t end up with ambiguous requirements, we make it easy to test, and we make sure that we actually give the users what they want?

The key is that we need to find common language that people in different roles can agree on so that we can all be on the same page. Users, BAs, QAs, and developers all think differently and speak different languages. So we need to find a better way to communicate and a better way to write requirements.

The way that we do this is using the Gherkin language, or Given/When/Then format. This goes something like this:

Feature: Transfer money between accounts
    As a user
    I want to be able to transfer money between active checking accounts
    So that I don't overdraw my account
    And I don't have to go to the bank to make the transfer

Scenario: Transfer positive amount
    Given a checking account "A" that has $100
    And a checking account "B" that has $50
    When I transfer $10 from account "A" to account "B"
    Then the transfer should be successful
    Then account "A" should have $90 
    And account "B" should have $60

At the top of the file we have background information about what the user wants to do and why they want to do it. Then we go step by step and identify all of the scenarios in the Given/When/Then syntax.

So what’s the big deal? Aren’t I just writing the same thing in another format? Yes, but the good thing about the Given/When/Then format is that it helps you find the holes in your requirements and think of all the details and edge cases. If you’re requirements are more in paragraph form with some bullet points, it’s really easy to miss things, even if you’re smart and trying to do a good job.

Essentially what we’re doing here is writing test scripts. As a developer, I want to know how QA is going to test my code. Not only that, I can take these scenarios and automate these tests with acceptance testing tools like SpecFlow or Cucumber or with plain old integration tests or unit tests.

What we did here was establish a common language between everyone involved. This will help BAs think in a detailed, logical way that ensures that they don’t miss things in the requirements. This will help developers because they won’t have to write code to hit a moving target or get stuck waiting for answers to requirements questions. This will help testers because all of their test plans get written up front before the development happens, so they don’t get stuck trying to figure everything out on their own after it’s been developed (and the BAs do a lot of their work because they write their test plans for them). Not only that, this gives the development team the chance to automate the tests. This will help users because once we get the Given/When/Thens done, we can take that to the user and ask them to confirm that we’re building the right thing. We’re still using their terminology, and the syntax will help them think of cases that maybe they hadn’t thought of initially.

If you write your requirements this way, if you get everyone involved up front, and if you write your code to make the tests pass, you should be completely done when you finish development. Because you wrote code against detailed requirements that were written without ambiguity in a manner that makes it easy to write tests for it, you should have no bugs (or at least very very few bugs). QA won’t have to spend lots of time afterwards trying to figure out how to test the feature. And hopefully the users are getting what they want.

In my last post, I talked about how acceptance tests can be more important than unit tests. But clearly there is another side of the coin.

The conventional wisdom is that you should have more unit tests than acceptance tests. This is how I’ve done it in the past (on pretty much every project other than my current one).

A common scenario is a web application that uses an ORM for data access with as few stored procedures as possible. Because I’m using an ORM, my data layer has very little code. I’ll probably write a few basic unit tests and integration tests to test what data layer I have, and I’ll write integration tests to test the stored procs that I have (usually only for queries that need custom SQL).

My data layer primarily contains a really simple repository class that lets me do CRUD operations and get sort of IQueryable so that I can write LINQ queries. (If you’re using Rails, you don’t have to build any of that because it’s all there for you out of the box.) I don’t want my repository methods to have any concept of business logic or specific LINQ queries. Those belong in the business layer. The exception would be any stored procs that I have to have. I’m want to have a really thin data access layer and a fat business layer (yet another reason to use an ORM instead of having lots of SQL code in stored procs).

The business layer is going to contain the meat of the code. This is where I’m going to have a lot of unit tests. Because I kept my data layer so simple, I’m not going to have as many classes to stub out, and when I do, I’m stubbing out simple methods like Save() and Get(), which have no business logic in them.

I also have controllers, which take information from the business layer and return some sort of view model. I’ll write unit tests that verify that that translation happens correctly.

If I have some hardcore JavaScript, I’ll write unit tests for that. If it’s really simple stuff like, “when this radio button is clicked, disable this text box”, I don’t write tests for that. But when you build something that has more logic, you’ll want to test it.

At this point, I’ve tested pretty much everything very extensively. I could add some acceptance tests, which could help me communicate with the business and automate some QA testing. But at the same time, I’ve been on projects where I had excellent unit test coverage and no acceptance tests and it came out just fine. Now, that project was a two-person project with no QA team, so we just trusted our coding abilities and accepted the risk and things came out just fine. The point of saying this is not to say that you should fire your QA team, it just an example of how you can structure your code in a way that will allow you to unit test the code and get it right pretty much all the time.

This is just one example of one project. Your project might be different, with a different architecture, a different team, and a different business. You need to figure out what works best for you.

The conventional wisdom on TDD and testing is that unit tests are the preferred kind of tests to write. They help you design your code, and they run fast. Integration tests and acceptance tests are important too, but you should have far fewer of these. They exist so that you can have some end to end tests and to help test things that you can’t easily test with a unit test.

You may have also seen the automated testing triangle diagram, which illustrates how you should have more unit tests than the other kinds of tests and how the unit tests are the foundation of your test suite.

The implication of all of these diagrams and ideas is that this is the best way to do it on every project. But is that true?

I would argue that there are situations where integration tests and acceptance tests are more important than unit tests.

But wait, you say, aren’t you the guy who is always speaking about TDD and the value of unit tests?!? Was that all a lie?

Once again, the correct answer is… it depends!

Take my current project for example. We are writing web services that do back end transaction processing. Our system is very configurable, so we can make the system work very differently based on some configuration tables in the database, and different configurations are used for different types of transactions.

We have a very well defined framework. Even before I start coding a feature, I know what classes I need to create because they were defined up front with a systems analyst. So I’m not really going to get any benefit from TDD designing my code because the design work is already done.

Because the system is so configurable, we can handle new kinds of transactions by inserting new configurations into the database. There is very little coding involved. Sure, I could write unit tests for the configurable components of the application, but unit tests can’t tell me if the system works correctly for my new configuration. The best way to do that is with acceptance tests.

Not only is our framework well defined, we follow the SOLID principles and have lots of small classes that work together. I could write unit tests for all of the small classes, and in some cases I do, but in
most cases I find that the unit tests aren’t testing very much. Not only that, they tend to have lots of stubs and mocks, which makes the unit tests very brittle. Usually the integration tests and acceptance tests are more brittle, in our case, it seems to be the other way around. Nothing seems like a waste of time more than updating an existing unit test that broke because you refactored a seemingly unrelated class, especially when you have a nice acceptance test that is working just fine.

I’ve just shot down many of the positive arguments for unit tests on our project. Because it’s very important for us to test that our system is configured correctly, acceptance tests become very
important.

Well, if I already have a big suite of acceptance tests, do I really need unit tests to test things that are well covered by acceptance tests? Maybe if I have some class that has many different permutations, but if there’s only a couple paths through a class and I’ve covered it with acceptance tests, then I’m just going to skip the unit tests.

We also use stored procedures to do most of our data access. You can’t easily unit test a stored procedure, so we need more acceptance tests and integration tests to take care of that.

We still have about twice as many unit tests as acceptance tests, but that tide is turning. In a couple months, we will probably have twice as many acceptance tests as unit tests.

It honestly took us awhile to convince ourselves that we weren’t doing it wrong based on the conventional wisdom. But when we thought about it it just made sense to do it the way we’re doing it.

The moral of the story is that there are always exceptions to the so-called “best practices”.

A few of us have been talking and have decided to start the Columbus ATDD Developers Group. If you’re in the Central Ohio area and you use tools like Cucumber and SpecFlow, believe that testing is not just for QA people, or if acceptance test driven development sounds interesting to you, then this group may be for you. This is a developer-focused group, so expect to learn things about how to write code using BDD testing tools, design patterns and practices, how acceptance tests mesh with your unit tests, and other developer topics. QA people are always welcome too, especially the technical-minded ones! Absolutely no experience is required, just come ready to learn.

Testing is not just for QA people, and BDD testing tools like Cucumber and SpecFlow have really helped me speak to business people, BAs, and testers with a common language and automate acceptance testing for my features. This hasn’t all been easy, and I’ve had to learn a new set of workflows, tools, and patterns. I found that I’ve asked a lot of questions about agile testing and BDD tools on Twitter and I’ve called people up when I’ve had questions on how to do things, and having those connections have really helped me out as I work through the learning process.

We are going to meet on the 1st Thursday of the month from 11:30-12:30 at various locations around Columbus. We’ll probably move the group around so that the same people don’t get stuck with the drive every month. This is an informal group so we’ll have open discussions as well as presentations and people showing their code. We want to hear everyone’s questions and ideas.

Our first meeting will be Thursday, March 1 from 11:30-12:30 at IGS Energy in Dublin. We will have an open space style discussion about Adopting ATDD in Your Organization. Some of us are far along down this path and some are just starting out, and we can all learn a lot from each other’s experiences. So come with your questions and ideas to share. And because IGS Energy is awesome, they will hook you up with free pizza and pop.

If you want to keep up with the group, we are in all the usual places:

• Blog (we’ll post meeting announcements as well as guest posts about ATDD topics!) – http://columbusatdd.wordpress.com
• Twitter – @ColumbusATDD
• Google group

I probably won’t post meeting announcements on my blog, so go follow the RSS feed on the blog.

Hope to see some of you there!