When acceptance tests are better than unit tests

The conventional wisdom on TDD and testing is that unit tests are the preferred kind of tests to write. They help you design your code, and they run fast. Integration tests and acceptance tests are important too, but you should have far fewer of these. They exist so that you can have some end to end tests and to help test things that you can’t easily test with a unit test.

You may have also seen the automated testing triangle diagram, which illustrates how you should have more unit tests than the other kinds of tests and how the unit tests are the foundation of your test suite.

The implication of all of these diagrams and ideas is that this is the best way to do it on every project. But is that true?

I would argue that there are situations where integration tests and acceptance tests are more important than unit tests.

But wait, you say, aren’t you the guy who is always speaking about TDD and the value of unit tests?!? Was that all a lie?

Once again, the correct answer is… it depends!

Take my current project for example. We are writing web services that do back end transaction processing. Our system is very configurable, so we can make the system work very differently based on some configuration tables in the database, and different configurations are used for different types of transactions.

We have a very well defined framework. Even before I start coding a feature, I know what classes I need to create because they were defined up front with a systems analyst. So I’m not really going to get any benefit from TDD designing my code because the design work is already done.

Because the system is so configurable, we can handle new kinds of transactions by inserting new configurations into the database. There is very little coding involved. Sure, I could write unit tests for the configurable components of the application, but unit tests can’t tell me if the system works correctly for my new configuration. The best way to do that is with acceptance tests.

Not only is our framework well defined, we follow the SOLID principles and have lots of small classes that work together. I could write unit tests for all of the small classes, and in some cases I do, but in
most cases I find that the unit tests aren’t testing very much. Not only that, they tend to have lots of stubs and mocks, which makes the unit tests very brittle. Usually the integration tests and acceptance tests are more brittle, in our case, it seems to be the other way around. Nothing seems like a waste of time more than updating an existing unit test that broke because you refactored a seemingly unrelated class, especially when you have a nice acceptance test that is working just fine.

I’ve just shot down many of the positive arguments for unit tests on our project. Because it’s very important for us to test that our system is configured correctly, acceptance tests become very
important.

Well, if I already have a big suite of acceptance tests, do I really need unit tests to test things that are well covered by acceptance tests? Maybe if I have some class that has many different permutations, but if there’s only a couple paths through a class and I’ve covered it with acceptance tests, then I’m just going to skip the unit tests.

We also use stored procedures to do most of our data access. You can’t easily unit test a stored procedure, so we need more acceptance tests and integration tests to take care of that.

We still have about twice as many unit tests as acceptance tests, but that tide is turning. In a couple months, we will probably have twice as many acceptance tests as unit tests.

It honestly took us awhile to convince ourselves that we weren’t doing it wrong based on the conventional wisdom. But when we thought about it it just made sense to do it the way we’re doing it.

The moral of the story is that there are always exceptions to the so-called “best practices”.