Ah, requirements. Gathering requirements is really hard. I’m not a BA, but like most developers, we’ve had to gather requirements at some point. You have to meet with users who might not have enough time for you or might not be able to make up their mind. Then you have to take their ideas and understand the business problem enough so that you can write up requirements. You have to try and find the balance between giving the users what they’re asking for and giving the users what they really need but don’t know it yet. It’s a fine line for sure.
I’m not going to sit here and bash BAs for lousy requirements. I’ve been the one who wrote the requirements only to have a developer mock the requirements (they probably didn’t know that I wrote them or that I was listening, but that’s neither here nor there). I tried really hard and I thought I did a good job. But nevertheless, there were things that I didn’t really think of that the other developers thought of.
Since we know that requirements gathering is hard, what can we do to make sure that we don’t end up with ambiguous requirements, we make it easy to test, and we make sure that we actually give the users what they want?
The key is that we need to find common language that people in different roles can agree on so that we can all be on the same page. Users, BAs, QAs, and developers all think differently and speak different languages. So we need to find a better way to communicate and a better way to write requirements.
The way that we do this is using the Gherkin language, or Given/When/Then format. This goes something like this:
Feature: Transfer money between accounts
As a user
I want to be able to transfer money between active checking accounts
So that I don't overdraw my account
And I don't have to go to the bank to make the transfer
Scenario: Transfer positive amount
Given a checking account "A" that has $100
And a checking account "B" that has $50
When I transfer $10 from account "A" to account "B"
Then the transfer should be successful
Then account "A" should have $90
And account "B" should have $60
At the top of the file we have background information about what the user wants to do and why they want to do it. Then we go step by step and identify all of the scenarios in the Given/When/Then syntax.
So what’s the big deal? Aren’t I just writing the same thing in another format? Yes, but the good thing about the Given/When/Then format is that it helps you find the holes in your requirements and think of all the details and edge cases. If you’re requirements are more in paragraph form with some bullet points, it’s really easy to miss things, even if you’re smart and trying to do a good job.
Essentially what we’re doing here is writing test scripts. As a developer, I want to know how QA is going to test my code. Not only that, I can take these scenarios and automate these tests with acceptance testing tools like SpecFlow or Cucumber or with plain old integration tests or unit tests.
What we did here was establish a common language between everyone involved. This will help BAs think in a detailed, logical way that ensures that they don’t miss things in the requirements. This will help developers because they won’t have to write code to hit a moving target or get stuck waiting for answers to requirements questions. This will help testers because all of their test plans get written up front before the development happens, so they don’t get stuck trying to figure everything out on their own after it’s been developed (and the BAs do a lot of their work because they write their test plans for them). Not only that, this gives the development team the chance to automate the tests. This will help users because once we get the Given/When/Thens done, we can take that to the user and ask them to confirm that we’re building the right thing. We’re still using their terminology, and the syntax will help them think of cases that maybe they hadn’t thought of initially.
If you write your requirements this way, if you get everyone involved up front, and if you write your code to make the tests pass, you should be completely done when you finish development. Because you wrote code against detailed requirements that were written without ambiguity in a manner that makes it easy to write tests for it, you should have no bugs (or at least very very few bugs). QA won’t have to spend lots of time afterwards trying to figure out how to test the feature. And hopefully the users are getting what they want.
I’m a huge fan of TDD, and one thing that is really frustrating is when I’m working with people who either don’t know how to write unit tests or don’t agree with the idea of writing unit tests. This is especially true in a static language because those people will design the code in a way that prevents me from writing tests.
Why is it that so many developers don’t care about testing? Most developers that I’ve worked with, whether I agree with their ideas or not, take pride in their work and generally care about their job.
I think that this goes back to the traditional divide between development teams and QA teams. On most teams, there is always pressure to get things done faster. In order to get things done faster, you have to either cut something out or get better at what you do. It’s much easier to cut something out, so inevitably the team will do what they can to cut development time. This means cutting out unit testing (or not taking the time to do it in the first place). Once you cut out unit testing, the next thing you throw out is good design principles and just get it done as fast as possible.
We all know that if you don’t write unit tests and you just throw in spaghetti code to get something done that you’re going to have a lot more bugs. It takes time for QA people to find bugs, reproduce bugs, write up bugs, and retest bugs. But why should the development team care? They were just told to go faster. Who cares how long it takes the QA team to test the application? Why does the development team care if it takes an inordinate amount of time to regression test the application?
The QA team knows what is going on, so they don’t trust the developers. They know that the developers are likely to write bugs, so they will need to be more thorough. The problem is that sometimes this means doing an inordinate amount of manual testing (which may or may not be possible).
This is what happens when development and QA are separate teams that don’t work closely together. Each one is only looking out for their own interests, not the interest of the project as a whole. In my opinion, it makes no sense to do it this way, but this how it’s done in many companies.
On my current project, we have broken down the walls. The QA team and the development team report to the same manager. The developers, BAs, and QAs sit together in the same area. Before we work on a feature, we all sit down together and decide what will be tested with unit tests, what will be tested with automated acceptance tests, and what will be manually tested. Because we’re all on the same team, I (as a developer) care about testing just as much as anyone else on the team. There’s no reason for me to not care about testing. Don’t I want the app to work too? If I write crappy code, it’s going to take longer for the team to get things done. If I don’t help automate regression testing, it’s going to take the team longer to manually test everything. I don’t want either of these, so I’m going to help out. The QA team is starting to realize that they can trust our automated tests (which they’re not used to), and we all agree that in the future they will do much less manual testing.
Because we’re on the same team, we’ve enabled developers to care about testing. If we want to get things done faster, I want to make sure that I automate as many tests as possible so that QA doesn’t have to manually test them. I might even pitch in to help with manual testing when that time comes. In the end, we end up with high quality software delivered in less time because we have very few bugs and we will test things in the most efficient way possible.
In most companies, there is a definite divide between the development team and the QA team. They don’t sit together in the same area, they don’t have the same boss, and they often aren’t even working on the same project at the same time. Sometimes the QA team doesn’t get involved until it’s time for the “testing phase.”
To me, this all makes no sense. I’ve been on teams like this, and the same thing usually happens — developers develop something, and when the QA team starts to look at it, they don’t know the first thing about the feature that was built. They try to come up with a test plan, but they don’t come up with all of the edge cases because they don’t know enough about the feature. Other times they come up with something that the developers hadn’t thought of, or they find bugs because the developers didn’t do anything to test their own code (even manually).
Before we go any farther we have to dispel two common misconceptions about the role of a QA team.
Some people think that your QA team is like a college professor giving you an exam. You learn about a feature from the BAs or the users, and then the QA team is there to verify that you understand everything after you’ve written the code. The QA team shouldn’t give you their test plans because then you might just write the code to make their test plans pass (heaven forbid!). That’s like your professor giving you the answers to the test, and that’s cheating. (I’m being sarcastic, of course.)
Other people think of the QA team like a group of external auditors. You don’t know when they’re testing or what they’re testing. They’re just here to catch you screwing up. They have to be extremely thorough and cover all the bases to make sure that nothing gets through, regardless of whether the development team has written any automated tests.
I would much rather work together with QA people. I would rather come up with the test plans together so that we can make sure that I automate most of their tests for them and build the right thing. I would rather have QA people involved up front so that they can help me think about how something should be tested. They also are very good are finding holes in requirements that no one else though of, which is very valuable to have up front before we right any code.
The hard part in all of this is getting people to work together who typically don’t work together. Usually developers and QA have different managers, and developers and QA people might not even be working on the same projects at the same time. You might need to get your management to work together to get people on the same teams working on the same projects at the same time.
We’ve been doing all of this on our current project and we’re really starting to see some exciting things. Much more on this to come!
As soon as you write code, bad things can happen. Your code can be defective, and it can break existing code. It’s our job to make sure that we don’t break things.
Not breaking things is not easy. You might end up with only one bug that sneaks through into production but that can be one too many.
The team I’m on is currently trying to figure out what our plan is for testing and QA. We have two concerns:
- How can we wrote code without defects?
- How can we make sure that our code changes don’t break existing code?
The second one is proving to be the difficult one. Our application is 4 years old, and a good chunk of the code was written by people who are no longer on the team. It’s really difficult at times to not break something when you don’t know enough about the system to know the ramifications of what you’re about to do.
When I develop new code, I try to figure out how we can validate that everything is working and complete. When possible, I want to write automated tests. In the Ruby world, I’m using Rspec and Cucumber to do this. I want to make sure that all of the code that I write is covered by a test if possible. Thankfully Rspec and Cucumber do not make this very hard at all, and we tend to test the crap out of things as a result.
But some things are hard to validate. For example, if you’re pulling a subset of data from an external database using an SQL query, how do you know that you’re pulling back the correct data? It’s hard (if not impossible) to write an automated test for this, and in some cases you have to coordinate with someone on the team that owns the database in order to test it. It’s really important not to ignore these tests and just say that it’s going to be too hard to test. Unless you’re OK with it being broken, you have to validate it.
As a developer, don’t think that this is just the responsibility of your QA team to figure out how to test things. Certainly your QA team should be involved in coming up with the plan for testing, but you might know of edge cases that a QA person wouldn’t know about, and you might know of technical constraints that might make testing hard. Too many defects are created because developers put the onus on QA to find everything. Remember, we’re all on the same team.
So now you go into production and you move on to something else, but you’ve created a lot of features that will need to be regression tested. Here is where I think it’s really difficult. How do you keep these regression tests up to date? For example:
- When you add a new security role to the system, do you update old test plans to test whether or not the new security role should be able to do something?
- If you change a validation rule, do you update old test plans that could be affected by the new validation rule? (This could mean adding new tests to an old test plan because you might now have two paths through something where you might’ve only had one test before.)
- When things change like this, how do you even know where you might need to update test plans?
I’d be interested in hearing how you all handle these situations. This is a situation where I’m really glad that I have automated tests (yesterday I was doing a big refactoring and my Cucumber tests pointed out a whole bunch of things that I had broken). But automated tests only solve the problem when you have a test for something. If you create new paths through the code that now don’t have a test, you need a way to find those scenarios and write tests for them.
