Lately I’ve been thinking about a whole team approach to testing, where we decide as a team how features will be tested and where we use the skillsets of the whole team to automate testing. We do this on our project, and this has led to a regression testing suite of ~2500 SpecFlow acceptance tests that automate almost all scripted QA testing and regression testing for our application.
We didn’t always do this. Originally there was no automated acceptance testing, but developers were diligently writing unit tests. Those unit tests are still around, but we don’t write many unit tests anymore. We start with acceptance tests now, and the acceptance tests cover all of the testing scenarios that need to be covered. Our application has well-defined design patterns that we follow, so the idea of TDD driving the design of our code doesn’t really apply. If the unit tests fail, we often just delete them because it’s not worth fixing all of the mocks in the unit tests that are causing them to fail, and we have acceptance testing coverage around all of it.
This approach does not line up with the conventional wisdom on automated testing. They say that you’re supposed to write lots of unit tests that run really fast to give you fast feedback, help design your code, and ensure the internal quality of your code. In the past, this is how I’ve always done it. In fact, many of them dislike Cucumber.
Cucumber makes no sense to me unless you have clients reading the tests. Why would you build a test-specific parser for English?
— DHH (@dhh) March 29, 2011
While TDD isn’t as mainstream as I would like, TDD is nothing new. Kent Beck was writing about it 10 years ago, and the original XP guys valued such things as unit testing, the SOLID principles, and things like that.
Automated acceptance testing still feels like a relatively new phenomenon. I’m sure people were doing it 10 years ago, but back then we didn’t have Cucumber and SpecFlow and the Gherkin language. Now I see a lot more people using tools like that to automate QA testing in way that uses business language and more maintainable code, rather than the old “enterprise” solutions like QTP.
Here’s what I’m getting at – I wasn’t there 10 years ago when Kent Beck was writing his books and the XP movement was starting, but it seems to me to be primarily an effort by developers to ensure the quality of their code through the effort of developers. I see very little talk of where QA fits into that process. There is some mention of QA for sure, but the general gist seems to be that developers need to write tests in order to ensure quality, and the best way to do that is to write unit tests. QA people typically don’t say that unit testing is enough because it doesn’t test end-to-end, so then what do they do? Manually test? Use QTP?
My question is this – if we think of testing as whole-team activity and not just a QA activity or a developer activity, will we arrive at the same conclusions as we did before?
I’m not ready to discount unit testing as a valuable tool, and I’m also not ready to say that everyone should do it my way because it worked for us on one project. But we have largely abandoned unit testing in favor of acceptance testing, and other teams in our department are doing it too. I write unit tests for things like extension methods and some classes that have important behavior on their own and I want to ensure that those classes work independent of the larger system.
We have 3 Amigos meetings in which one of the things we do is develop a set of acceptance tests for a feature before any code is written. We usually decide at this point (before any code is written) that most or all of these scenarios will be automated. We write the acceptance tests in SpecFlow, I watch them all fail, and them I write the code to make them pass. I follow the patterns and framework that we have set up in our application, so there aren’t many design decisions to make. When my acceptance tests pass, I am done.
Where do unit tests fit in there? If my acceptance tests pass, then I’m done, so why spend more time writing duplicate tests? Also, with acceptance tests, I’m not dealing with mocks, and more importantly, I’m not fixing broken unit tests because of broken mocks. If you follow the Single Responsibility Principle (which we try to do), you end up with lots of small classes, and unit tests for those classes would be mostly useless because those classes do so little that it’s hard to write bugs and each class does such a small part of the larger activity.
There is an obvious trade-off here – my acceptance tests are not fast. I’m just testing web services (no driving a browser), so all ~2500 tests will run in about an hour. But we accepted this trade-off because we were able to get things done faster by just writing the acceptance tests, which we were going to do anyway to automate QA testing. The end result is high quality software with few bugs, not just because we have tests, but also because we communicate as a team and decide on the best way to test each feature and what it is that needs to be tested, and then we find the best way to automate the testing as a team.
Again, I’m not ready to say that this way is the best way for every project, and I’ve seen each approach work extremely well. I just wonder if the conventional wisdom on testing would be the same if we thought of it from the perspective of the whole team.
I’m a big fan of board games, and I have invested many hours of my life playing board games. I’m not talking these “group fun games” like Cranium or Guesstures, I’m talking about games that have more strategy like Settlers of Catan, Ticket to Ride, and Dominion.
Each of these games has it’s own strategy, but I’ve found that in all of these games, the most important strategy is to have a plan for winning. Not only do I have to have a plan for winning, I need to do it fast. I’ve won many games where I came from behind because the person who was ahead got comfortable with the lead and didn’t keep pushing forward.
The same idea applies to software development. I don’t like when command-and-control managers try to push unrealistic timelines on development teams. But at the same time, every business could use everything that you’re doing for them yesterday, so we need to find a way to go faster.
This means that I need to make the best use of my time, all the time. This especially applies when things are a little slower. We’ve had times where we’ve had weeks where there was nothing pressing to be done and we decided to tackle technical debt in the code. This is one of the most important times on a project because this is your chance to come up with a way to do things better and faster.
This is when you need to have a plan for winning. You need to determine what things you can do that will help you write better software faster. You definitely don’t want to squander the opportunity, because who knows when another one like it might come along.
We are in a race against time. Any time wasted means lost revenue, lost productivity, missed market opportunities, and higher development cost. All of these things are critical to the business. Imagine that we were writing an app for your company with your money. Would you do things differently?
Everything that you do on a project should be done with a sense of urgency. Not a fear-based pressure that leads you to make bad decisions, get flustered, and cut corners, but a conscious effort to maximize the return on the investment of your time (and yes, your time is expensive). This shouldn’t have to come from your manager or some outside source, this needs to come from inside you.
In our fast paced culture, we are constantly looking ahead to the next big thing. We think about some magical someday when things will be better than they are now. Then when we get there we’re still looking ahead to something else.
I’m all for setting goals for the future and reflecting on the past. The danger in all of this is that we might just miss out on the present.
Recently I’ve been trying to live in the moment and be present in the present. I want to enjoy the people and opportunities that I have in front of me right now. I want to actually stop what I’m doing in my busy day and turn around and listen to someone who wants to talk to me. I want to spend less time looking for the next big thing and spend more time taking advantage of what’s in front of me.
You can’t take anything for granted in life because things change too fast. Never again will things be the same as they are right now. A year from now you might be working with different people, friends might move away, and your kids will be a year older. Sometimes it seems like things will always be the same, but nothing lasts forever.
So I find time to enjoy today. I’ve started to go to lunch with people more often because I want to enjoy them as much as I can. I had a great time in college, and I would give a lot to be able to have one day with those people all in the same place again doing the same things, but those days are gone. I hope that I’m saying that same thing about 2012 in a few years, and I don’t want to regret not taking advantage of it while I still had the opportunity.
While it’s wise to study the past and plan for the future, don’t forget to enjoy what you have now. Because you’ll never have another today.
I saw this in a job posting the other day, describing the work environment at the company:
We pair a lot, but not all the time. We test a lot, but not all the time. The key is being able to explain your practices with rational argument.
I love this. There are practices like testing that I hold in high regard, but nothing should be done all the time. As the posting says, “the key is being able to explain your practices with rational argument.”
Can you explain why you do something like TDD? That’s probably easy for you. But can you explain when you shouldn’t use that practice? That’s a little harder to do, and maybe because you’re holding onto some ideal that says you should test all the time or have 100% test coverage.
For every idea or practice that you find valuable, you should also be able to explain when it doesn’t apply. If you can’t explain when not to do it, I would say that you don’t really understand when and why you should use it.
What do you do when you get stuck on something at work? Do you ask for help? Or are you too afraid to bother someone else with your problem?
This is one of my pet peeves. When you’re writing software, gathering requirements, testing, or whatever you are doing, at some point you are going to get stuck on something. You spend a little time trying to figure it out, but you can’t solve the problem.
At this point, many people are afraid to ask for help. I don’t think that it’s because they’re too proud or lazy to do so, they just don’t want to inconvenience me because I “look like I’m busy” or they “don’t want to bother me”.
I may actually be busy and I maybe I even don’t want to be bothered, but if I can spend one minute answering someone’s question and save them 20 minutes of figuring it out (and the headaches that come with it), I’m going to do take the time to do it. Even though it’s an interruption, that time spent helping someone else is a good investment that will pay off because hopefully that person will no longer be stuck. If I really am so busy that I can’t help, I’ll just tell people that and they usually understand, and then I’ll try and follow up with them later.
I feel that in general we are so afraid of inconveniencing other people, even our good friends. I have good friends who have young kids like I do and they will go multiple years without going on a date without the kids. I’ve offered several times to watch their kids for them (which is not that hard to do since their kids will occupy my kids), yet to this day no one has ever taken me up on it. We’re talking about people that we are good friends with. Yet they are so afraid of inconveniencing us (even though we were the ones to offer to babysit) that they won’t do it.
I don’t really know where this comes from, but it’s frustrating. Several people working together are going to be more efficient that individuals working on their own, and that’s because we can work together to achieve something that we couldn’t achieve on our own. But that won’t happen if we aren’t willing to bother someone else with our problems.
I frequently try and tell people on my team that it’s totally OK for them to bother me. I think my business analyst interrupted me at least 10 times the other day to ask me questions. This made me excited because she wasn’t afraid to inconvenience me with her questions, and as a result she probably got a lot more done today than if she had just tried to struggle through it herself.
Software development is a very collaborative activity, and we will be way more efficient if we ask others for help. So ignore that little voice in your head and go ask someone for help, and don’t feel bad about it.
These days, there is a certification for everything. Almost every discipline in software development has several different certifications that you can get if you attend a training class and then take a test that verifies that you understand the book definition of a process or language. This totally misses the point.
In some industries, certification matters. I’m glad that my doctor has to go through 12 years of school and thousands of dollars in student loans in order to get his M.D. If someone can make it through that, I have confidence that they know what they’re doing.
The bar for software certifications is much lower. All you have to do is pay a few thousand dollars for a training class and then take a test that shows that you paid attention in class and understand the basic concepts.
What do we think we get out of these certifications? A basic understanding of the principles, maybe. The problem is that those things are the easy part of software development.
Here are some of the things that I actually look for when hiring people:
- Ability to listen to users
- Understanding your team members
- Motivation and purpose
- Ability to think outside the box (as opposed to being certified that you know that you how to think inside the box)
- Learning ability
- Problem solving skills
I feel that one of my greatest strengths is the ability to make it up as I go. Sure, I might have my Agile bag of tricks, but every project and environment is so different. When we inevitably encounter a problem, the hard part is finding the right solution for it, and no formula or certification is going to tell you the right solution to every problem.
Certifications aren’t all bad, if you spend a week learning learn how to be a “certified Scrum developer”, you’ll learn TDD and good OO practices and other things that will help you succeed. We can argue about whether it’s worth the money, but you are going to learn some good stuff.
The point of all this is that (in my opinion) some of them most important qualities in a team member are less quantifiable things like attitude, passion, and the ability to think outside the box. These are the things that I want to strive for, and the only “certification” that you can get for this is experience.
If you’re a developer, do you have your own personal development database that you can recreate at the touch of a button?
If you said no, you’re not alone. In fact, I’ve been in that situation on many projects, where the entire team shares one big development database that is a backup of production, and all schema changes need to be coordinated with the other developers on the team. Usually things are this way because back in the day someone didn’t create a way to have local databases.
We were in the situation and we made the investment to have someone spend a week or two taking a copy of production, deleting all of the data from tables that weren’t lookup tables, and then creating a way to run all of the new SQL scripts against the small baseline copy. Now I can recreate my development database at the touch of a button.
The benefits of this are obvious, but since many people haven’t made this investment yet, let me list them anyway:
- I can hack up my database and not worry about affecting other developers
- I can hack up my database and then start over if I screw something up (as opposed to piece things back together so that the rest of the team can keep working)
- Automated tests that run against the database run MUCH faster
- No more confusion when someone else on the team changes some important piece of data without telling you
- I can check out any revision of the code and create the database that existed at that point in time
If you don’t have a local, recreateable database yet, make the investment. You will reap the benefits many times over and you’ll be so glad you did.
One of my favorite tricks around changing how a team does something is to get other people to drive the change for you.
This especially applies if you’re coming into an organization as an outsider. Most people are initially somewhat skeptical of outsiders and you need to earn their trust. This sometimes takes a long time, and you might want to change some things before you’ve built up the political capital to drive it yourself.
When I started on my current project, all of the QA testing was done manually, and QA people weren’t involved in the process at all until after the code had been written. As a result, they would get features to test and not have any idea of what they had to test, and many times they couldn’t trust that the requirements were even up to date.
The solution that I was hoping for was to get QA people to meet with developers and BAs before the code was written so that they could come up with a test strategy and acceptance criteria. That way they would have input up front, and developers could automate all or most of the acceptance tests, which would save QA a lot of time. But I didn’t want to bring it up myself.
So I would ask questions in meetings. We all agreed there was a problem, so I would ask everyone what ideas they had to fix it. Eventually someone would give the answer I was looking for. “That’s a great idea!”, I’d say, “How do you think we should do that?” Now it wasn’t my solution, it was someone else’s. That person feels good because they came up with a good idea (that eventually ended up working), and since that person is actually on the team and doing the work, they might have more credibility than an outsider like me.
I would much rather have someone else get the credit for coming up with the ideas, because I think that it gives the idea a better chance of success, and when it works, it encourages people to try and think of more ways to improve things. If one person comes in and tries to drive and enforce all of the change, it’s very easy for people to discredit that person (maybe even just in their mind) and try and fight the change. But when the ideas are coming from multiple different people on the team, now you have more and more people buying in because they’re the ones coming up with the ideas!
When I think of a “coach”, I think of someone who helps other people find the path to success. You can try to push them in that direction and they might turn around and fight you instead. Or you can help them use their own ideas and expertise that they already have to find their own way.
I am a software developer and technical leader in Columbus, OH, specializing in software solutions, project leadership, and Agile coaching and training in a wide range of industries and environments. Find out more here...