We had a week-long debate awhile back about whether or not it’s OK to modify your production code in order to enable automated acceptance testing. I’m not talking about using dependency injection, interfaces, etc. to allow you to mock things in unit tests. I’m talking about modifying application code solely to help your automated acceptance tests.
There are many ways this can be done, some of which we’ve done:
- Creating a SystemTime class, which is like DateTime except that we can set what “Now” is, so we can change time in tests
- Adding optional parameters to stored procedures solely so that we can have them only operate on a subset of data in an acceptance test instead of operating on the entire set of data in the database
- Adding extra HTML attributes so that automated tests can find elements on a page easily
To me, modifying production code to help us do automated testing is no big deal. First, if our goal is quality, I don’t think it matters how we get there. After all, we own the code base and tests so there aren’t any real restrictions on what we can do with the code or the tests as long as the end product is good.
Second, developers and QA are on the same team, and we work together quite closely, so we should do what we can to help each other out. So if we can make a minor change to the application code to save us a lot of time developing or running automated tests, then to me it makes sense to do so.
This goes back to my assertion that we need to stop thinking of QA like external auditors that have to take the application just as it is without talking to the developers and act as the independent quality police. We need to all work together to ensure quality, both developers and QA. Developers are just as responsible for quality as QA. If we place all of the responsibility for quality on QA, then developers will care less and less about quality, and you end up with shoddy code with lots of bugs (and usually no tests). I’d rather treat testing as a whole-team activity and structure the application to make testing as easy as possible.
Your production code is there for your business. If it’s important for your business to automate acceptance tests, then you better damn well be willing to alter your production code for that purpose. Sure, as you stated, you have to keep the quality level. But unless your production code is “the most efficient proprietary algorithm to calculate prime numbers”, then you do it. Just make intelligent decisions when doing it, as well.
There are ways to mess with DateTime other than abstracting it behind a service. I think the approach would depend on how you expect acceptance tests to set the value from the “other” side of the UI, while stopping normal users from doing the same. It’s a similar situation for the stored procedure parameters. I would think that running the acceptance tests in a separate QA environment that only has a subset of the data to begin with would be a better approach there. As for the HTML atributes, I’m 100% behind that one. There’s no harm in an extra attribute.
Mel,
I don’t really feel like I need to enforce that no one will change SystemTime.Now in actual production code… I’d rather just trust my developers. Or maybe make it a two step process (set InTest=true, then you can set SystemTime.Now) so that people can’t accidentally do it.
If I have acceptance tests for all of my code, then I should catch any problems anyway.
Having a divide between dev and QA encourages both teams to search out ways of maximizing their own throughput and minimizing their own cost, sometimes in ways that are detrimental to the other. Unifying the teams (theoretically) aligns everyone on the same goal of reducing cost and maintaining quality.
I’ve not yet met the QA person that hasn’t been made more valuable by bringing them closer to the dev team.
And, to your main point, I find it hard to believe that people would argue AGAINST changing production code to make it easier to test. Even if changing that code has risks, the changes are being made TO SUPPORT TESTING which should more than offset that risk through faster feedback and better coverage!
I agree with you that it seems fine to alter production code to provide seams for automated testing. My one caveat, personally, would be that I wouldn’t want the production code to have a context dependency — to be “aware” that it was in some kind of “test mode”. So adding decorations to code/markup, wrapping a class that provides the concept “now”, etc are all things that I think would b helpful — I just wouldn’t want to see some concept in production code like if(AmIBeingTested) { ….
But then again, I think context dependencies of any sort usually tend to be bad form.