In a month or so, my co-worker and I will be wrapping up the project that we have been working on for the last 15 months. It’s a website for a company in the construction industry that bids on jobs and then tracks the progress of the jobs, purchase orders, billing, and everything else they need to run their business. We used ASP.NET MVC, Fluent NHibernate, AutoMapper, and SQL Server 2008.
We practiced test-driven development from day one. We wrote tests for our .NET code, tests for our JavaScript code, and tests for our SQL code. Right now we have over 14,000 tests, and I think we can break the 15,000 test barrier in the next month. I can run them all in about 5 minutes.
I cannot tell you how invaluable these tests have been. First of all, our application deals with money. The users will input a bunch of data about a job and then our application will tell them how much to bid on the job. We cannot afford to have bugs in our code that would miscalculate the amount to bid on a job, because that would lead either to over-bidding (in which case they would win very few) or underbidding (in which case they would win jobs and take a big hit). It just has to work.
Second, we did not have a QA team on this project, it was just the two of us developers. Frankly, I don’t have time to go back and manually test stuff in the app or regression test it when we need to make a change and deploy something. We do have bugs from time to time, but we haven’t had any critical bugs.
Because of our tests, we have been able to get as close to continuous deployment as I would feel comfortable with. On average, we deploy 2-3 times a week. When we go to do a deployment, I usually go through the site and manually test the new features that we are about to release. If I don’t find any problems, I run our deployment (which is all automated), and 5 minutes later, the changes have been deployed. I never go back and regression test old features or stuff in areas that we didn’t change.
Since the whole test and deploy process takes only about 15 minutes, users get their changes quickly. They don’t have to wait until later in the week, or until our next scheduled release. We have reduced the cost of change to pretty much just the time that it takes to code the changes.
All of this is possible because we were diligent about test-driven development and writing good unit tests for everything. The reason that we don’t spend time regression testing is that we don’t expect anything to be broken, and it very rarely ever is. This means that we can spend more time delivering business value and less time ensuring that we didn’t break something we wrote a year ago.
I’m not writing this to say how awesome we are as developers, because anyone can write tests and have the same kind of success. Also, we could’ve cheated and not written tests and ended up with potentially costly bugs and more time spent regression testing.
This is why I practice test-driven development. You end up with well-designed code, you drastically reduce bug counts, you can release more often, your codebase stays under control, and you have a lot less stress.
Impressive. What’s your break down for unit/integration/ui/javascript/sql/etc test counts?