Most people I know put their unit tests in files that mirror the folder structure and filename of the actual class that is being tested. So if I have app/models/order.rb, I’ll have spec/models/order_spec.rb. The tests in the order_spec.rb file will test the code in the order.rb file. Every project I have been on, whether Ruby or .NET, has done it this way.
But have you ever thought about why we do it this way?
We do it this way on our project and I keep running into two problems. The first problem is when I am about to modify some existing code and I want to know what tests will test that portion of the code. The first place I look is the corresponding test file (based on the folder structure or filename), but that doesn’t always give me all of the tests for that functionality.
The second problem is when I refactor some existing code and the refactoring spans multiple classes. Now I have broken tests all over the place and it’s hard to reconstruct the tests so that they test the same business concepts that they were testing before. Often times the tests were testing a portion of the system of classes that I am refactoring, but were not as encompassing as they should’ve been.
Here’s the thing — when I write tests, I’m typically using the Given/When/Then style of writing tests, even in unit tests using frameworks like RSpec. I’ll have test code that looks like this:
describe "When completing an order" do
it "should set the status to In Process" do
# test code here
end
end
That code tests functionality having to do with orders and is going to help me ensure that my code performs some business function correctly. But if you look at that code snippet, you don’t know what classes I’m testing. Yeah I know, it’s just an example and I left out those details. But the point is that it doesn’t matter what classes I’m testing. What matters is that my tests are testing that my code performs a certain business function.
That being said, why are we grouping our tests by file? Wouldn’t it make much more sense to group our tests by business function instead?
I already have ways to find tests for a given class. I can search my code for the class name, or if I’m in .NET I can do Find Usages and have a little window pop up that tells me everywhere a class (or method) is used.
If my classes were grouped in folders by business function instead, I get the following benefits:
- I can see what tests exist for a given business function
- It encourages me to write tests that test business functionality, not test data structures (classes, methods, etc.)
- I can put all kinds of tests in there (unit tests, Cucumber tests, even manual test plans) — all in one place, all checked into source control
- My tests document business functionality instead of documenting a class
Remember, code constructs like classes and methods are just a means to an end, our goal is to write software that provides business value and performs specific functions. So I might as well organize my tests accordingly.
(Disclaimer: I have never actually tried organizing tests this way. It makes sense to me and I think it would work great, but I might try it and find out that it doesn’t work. But if anything, maybe I’ll start a good discussion.)
It’s my hope that we’ll be organizing tests by business function in our upcoming project at my job. It just seems like it makes so much more sense! I agree too, should be much easier to maintain and navigate.
Matt Gawarecki — June 28, 2011 @ 7:59 am
Why do we group our tests by file? Easy: You can explain it in one sentence and there is zero ambiguity as to where they go. I agree that business functionality might make more sense — particularly if BAs are writing specs — but developers, left to their own devices, tend to go with straightforward rules which don’t require a lot of thought.
That’s leaving aside environments like Rails/RSpec/Autotest/Watchr, where the filenames actually have meaning to the test.
Craig Stuntz — June 28, 2011 @ 3:41 pm
We need to have predictability in our test infrastructure. Keeping test methods in a known place under a known class name makes test creation and maintenance a straightforward process. For unit tests, we do not want to test cross-cutting concerns - instead we would like to isolate our method under test and test one atomic feature per test method.
On the other hand, integration or system-wide tests should be written the way you describe. I also think BDD-style test methods are only appropriate for system-wide integration tests as internal implementation and the actual location of code should not matter in that case.
Arsalan Ahmed — June 30, 2011 @ 2:56 pm