Metrics, and why I usually dislike them

Every team, company, and IT department wants to collect metrics these days. Sometimes metrics can be used for good purposes (e.g. are we on schedule), and some can be used for bad (e.g. let’s come up with some way that we can attempt to quantify how good our teams are so that we can compare them).

Let’s just get this out of the way — I really dislike metrics when they are used incorrectly. I think metrics are a good way of measuring your own progress internally. For example, I want to know if I’m on schedule to meet a target date. If the team is estimating work, I want to know how the actual time spent on the features compares to the estimates so that we can get better at estimating. I want to know how much time team members are spending on different activities (e.g. requirements gathering, development, production support) so that I can make future plans and staff the team accordingly. I might want to measure cycle time (the amount of time from when we start working on a feature to the time when the feature is deployed or ready for deployment). In all of these cases, two things are true: I’m not publicizing the metrics outside our team, and all of the metrics are measuring the performance of the team, not the performance of individuals.

I may use the metrics in order to know what message to send to a project sponsor. For example, if I can determine from our metrics that we’re ahead or behind, I would let the project sponsor know how ahead or behind we are. I might use cycle time to come up with a date when we think we can have a feature completed.

What I don’t want to do as a team lead is share any of the raw data with the people above me (unless I can trust them). Why? Because I don’t want to give them an opportunity to misinterpret the metrics and manage my team for me. If I’m leading a team, it’s my job to lead the team and it’s the team’s job to deliver. Too often, the higher-ups will use metrics in the wrong way.

This is what happens when your manager comes to you and says that you have to cram in some extra scope and cut out testing. They saw a metric (time for a developer to complete a feature) and made a decision based on it. They essentially are saying that “development time” is the most important metric, when in reality, what’s more important is the time to do all of the work (requirements, development, and testing). They are stepping in and managing the team and choosing a metric that you gave them (a development estimate).

This is why you have to choose your metrics wisely. What if you gave estimates to your project sponsor in terms of how many person-hours it takes? Let’s say that feature X will take 4 hours of BA work, 2 days of development, and 1 day of testing. In the end, that means that 3.5 days of people’s time will be needed. You probably know how much available BA and QA capacity you will have just like you know how much development capacity you will have. So why not portray it as one big bucket of capacity? Maybe you still break down the time and give that estimate by role. You run the risk of the sponsor trying to lop off the testing, at which point you have to earn your salary and explain to them why you need that much testing time (we wouldn’t say that we needed it if we didn’t need it, right?). If they don’t make QA people available to you, then use developers or BAs to do the QA. Either way, you need 3.5 hours of people’s time to do it.

Notice what I did differently — I gave a whole-team estimate. I didn’t just say how long it takes to do the development. That opened the door for your project sponsor (who probably doesn’t know how to manage your team) to staff the BA and QA team for you.

To me, the kinds of metrics I want to expose outside my team are things like features delivered, money saved/earned for the company, and things that show business value. Ultimately, this should be what matters to the business. These aren’t always easily quantifiable (which makes it difficult), but this should be what project sponsors and executives should ultimately care about. In fact, I would keep a list of achievements that you know have business value and report that up. Your project sponsor/manager would love that because they’ll turn around and report those same things to their boss. Now you’re speaking in the language that they should be caring about (cost savings, business value), and you’re free to manage the team the way that you know how.