Iteration Management – Post #6 – Comparing estimation methods

This post is a part of a series of posts about iteration management. If you want to start from the beginning, go here.

Some teams like to estimate in hours, and other teams like to estimate in “story points”. Other people simply try to break down all work into small tickets and just count the number of tickets completed. Other people don’t estimate at all. Which way is best?

Of course, it always depends on your situation, and people have views and opinions that are all over the place. I’ll give you my opinion and try to accurately represent all viewpoints, but this is just my opinion, so feel free to disagree with me.

Estimating in actual time (hours, days, etc.)

Pros:

• Estimating in hours makes sense to a lot of people, and they’re used to doing it.
• If you’re on a project and the team understands the application, the business, and the subject matter very well, you might be able to accurately estimate how long something will take (especially if you can break it down into small tickets).
• Works well when you can have the people who are going to do the work do the estimating. (This works best when you can estimate tickets shortly before you’re going to work on them so that you know who will be working on them.)

Cons:

• Estimates can vary from developer to developer. One developer who has been on the team for awhile and works quickly might estimate something at 4 hours, but a slower developer who hasn’t been on the team for awhile might estimate 16 hours. Both estimates are correct (for that person). This is OK, but if you don’t know who is going to be doing the work and the 16 hour developer inherits the 4 hour estimate, you’re going to have a problem.
• Doesn’t work well if you don’t know enough about the application, the business, or the subject matter, which makes it really hard to come up with an estimate in actual time that has any chance of being accurate.
• If you track velocity over time, it should be relatively consistent in terms of hours of work completed, but this won’t show if the team is getting faster (or slower) over time because the team’s estimates of a certain type of ticket will decrease over time. If you estimate in story points, a certain type of ticket will always have the same point value and you will see an increase in velocity over time.

Estimating in story points

Story points are arbitrary values that are given to a feature and then all other features are estimated relative to how difficult each task is to the previous tasks that have been assigned a specific story point value. People measure “points” differently depending on who you ask – some people say points are a measure of time/effort and others say that points are a measure of complexity. There’s no one right way, which is the point of this post.

One popular method is to use the Fibonacci sequence numbers as the point values that you can use in your estimates (e.g. 1,2,3,5,7,11,13,17,23). This makes it a little easier because you don’t have to agonize over whether something should be a 13 or a 14.

Pros:

• Works really well if you are estimating a project and you don’t know much about the application, the business, the technology, or the subject matter. In these cases, estimating in hours is a complete stab in the dark, but it’s much easier to compare the effort or complexity of one feature vs. another. (This happens a lot when you work for a consulting company and your company wins a bid for an entire project.)
• People aren’t very good at estimating in actual time, but they’re much better at estimating the size of a task relative to other tasks that have specified point values.
• You might have one developer who would estimate a task at 4 hours and another developer who estimates the task at 16 hours. But both developers might estimate the task as 2 points because they’re measuring the task relative to other tasks.
• Tracking velocity has more meaning. If the team gets faster over time and now a 2 point ticket takes 4 hours instead of 6 hours, you should see an increase in velocity over time, which is good for the team (especially if you’re trying to justify doing Agile in an environment that is new to Agile and is wondering if it’s worth it).

Cons:

• It can be awkward to estimate in story points, especially for new people who don’t know how the points have been allocated in the past. Other people have a hard time thinking in terms of story points when estimating a ticket in hours might seem easy.
• Team members and management tend to try to convert story points to hours, at which point people start to feel like maybe they should just estimate in hours anyway.
• Since a story point is arbitrary, it can become really easy to game the system. For example, management wants the team to increase velocity. Suddenly what would be a 2 point ticket now gets estimated as 3 points! (I realize that you could game the system if you were using hours as well, but it’s a little harder to do with hours since hours aren’t an arbitrary measurement.)
• Management at some point is going to ask for an estimate (read: commitment) where you have to specify a date when you need to be done, so you’re probably going to have to translate points to hours.

Counting tickets

Pros:

• Estimating feels like a waste of time and the estimates are often wrong anyway. If the tickets are all small and relatively close in size, the differences in size all average out anyway so we can just count the number of tickets and it will come out close enough. (This, of course, is dependent on being able to break down the work into small tickets of about the same size, which is hard to do on some projects.)
• No more estimation meetings!

Cons:

• Doesn’t work well if you aren’t able to break down the work into really small tickets.

An alternative way to count tickets is to have a small number of sizes of tickets (e.g. small/medium/large). This is similar to estimating in story points, but it’s less granular. You still count tickets, but each ticket has a point value (for example, small = 1, medium = 2, large = 3).

No estimation

Pros:

• Works well if you just work the tickets as they come to you and you don’t have to do long term planning (e.g. handling production support, ongoing maintenance).
• Works well if a good portion of your work is not known up front and just comes to you when it needs to be done.

Cons:

• Won’t work if you’re working on a project where management needs to know when it will be done.

More on story points

I often hear complaints from people that like story points. They say, “Management keeps trying to translate story points to hours!” My response to that is, what’s wrong with that? If you’re on a story point project, eventually you’ll have enough data to show that a 1 point ticket typically takes X hours on average, and so on. Isn’t that a good thing? Aren’t you better off if you’re able to translate story points to hours? Who wouldn’t want to have that knowledge?

This doesn’t mean that you can’t continue to estimate in story points, but remember, the whole point of estimating is ultimately to tell management or the business when you are going to get work done. There is still value in story points (more meaningful velocity charts, easier to estimate in some cases, etc.), but at some point the translation to hours is probably going to be necessary.

Use your brain

I find that people are very passionate about this topic, and the most passionate ones either think that you should always estimate in story points or never estimate in story points. I don’t like to use words like “always” or “never” because you can always find a situation where it makes sense to do things a certain way.

This is where you come in. You can read all of the books and blog posts that you want, but ultimately you need to use common sense and find the estimation method that works best for your team. There’s no harm is trying something and switching to something else later. The important thing is to do more of what works and less of what doesn’t.

Read the next post in this series, Data analysis.