This post is a part of a series of posts about iteration management. If you want to start from the beginning, go here.
Some teams like to estimate in hours, and other teams like to estimate in “story points”. Other people simply try to break down all work into small tickets and just count the number of tickets completed. Other people don’t estimate at all. Which way is best?
Of course, it always depends on your situation, and people have views and opinions that are all over the place. I’ll give you my opinion and try to accurately represent all viewpoints, but this is just my opinion, so feel free to disagree with me.
Estimating in actual time (hours, days, etc.)
- Estimating in hours makes sense to a lot of people, and they’re used to doing it.
- If you’re on a project and the team understands the application, the business, and the subject matter very well, you might be able to accurately estimate how long something will take (especially if you can break it down into small tickets).
- Works well when you can have the people who are going to do the work do the estimating. (This works best when you can estimate tickets shortly before you’re going to work on them so that you know who will be working on them.)
- Estimates can vary from developer to developer. One developer who has been on the team for awhile and works quickly might estimate something at 4 hours, but a slower developer who hasn’t been on the team for awhile might estimate 16 hours. Both estimates are correct (for that person). This is OK, but if you don’t know who is going to be doing the work and the 16 hour developer inherits the 4 hour estimate, you’re going to have a problem.
- Doesn’t work well if you don’t know enough about the application, the business, or the subject matter, which makes it really hard to come up with an estimate in actual time that has any chance of being accurate.
- If you track velocity over time, it should be relatively consistent in terms of hours of work completed, but this won’t show if the team is getting faster (or slower) over time because the team’s estimates of a certain type of ticket will decrease over time. If you estimate in story points, a certain type of ticket will always have the same point value and you will see an increase in velocity over time.
Estimating in story points
Story points are arbitrary values that are given to a feature and then all other features are estimated relative to how difficult each task is to the previous tasks that have been assigned a specific story point value. People measure “points” differently depending on who you ask – some people say points are a measure of time/effort and others say that points are a measure of complexity. There’s no one right way, which is the point of this post.
One popular method is to use the Fibonacci sequence numbers as the point values that you can use in your estimates (e.g. 1,2,3,5,7,11,13,17,23). This makes it a little easier because you don’t have to agonize over whether something should be a 13 or a 14.
- Works really well if you are estimating a project and you don’t know much about the application, the business, the technology, or the subject matter. In these cases, estimating in hours is a complete stab in the dark, but it’s much easier to compare the effort or complexity of one feature vs. another. (This happens a lot when you work for a consulting company and your company wins a bid for an entire project.)
- People aren’t very good at estimating in actual time, but they’re much better at estimating the size of a task relative to other tasks that have specified point values.
- You might have one developer who would estimate a task at 4 hours and another developer who estimates the task at 16 hours. But both developers might estimate the task as 2 points because they’re measuring the task relative to other tasks.
- Tracking velocity has more meaning. If the team gets faster over time and now a 2 point ticket takes 4 hours instead of 6 hours, you should see an increase in velocity over time, which is good for the team (especially if you’re trying to justify doing Agile in an environment that is new to Agile and is wondering if it’s worth it).
- It can be awkward to estimate in story points, especially for new people who don’t know how the points have been allocated in the past. Other people have a hard time thinking in terms of story points when estimating a ticket in hours might seem easy.
- Team members and management tend to try to convert story points to hours, at which point people start to feel like maybe they should just estimate in hours anyway.
- Since a story point is arbitrary, it can become really easy to game the system. For example, management wants the team to increase velocity. Suddenly what would be a 2 point ticket now gets estimated as 3 points! (I realize that you could game the system if you were using hours as well, but it’s a little harder to do with hours since hours aren’t an arbitrary measurement.)
- Management at some point is going to ask for an estimate (read: commitment) where you have to specify a date when you need to be done, so you’re probably going to have to translate points to hours.
- Estimating feels like a waste of time and the estimates are often wrong anyway. If the tickets are all small and relatively close in size, the differences in size all average out anyway so we can just count the number of tickets and it will come out close enough. (This, of course, is dependent on being able to break down the work into small tickets of about the same size, which is hard to do on some projects.)
- No more estimation meetings!
- Doesn’t work well if you aren’t able to break down the work into really small tickets.
An alternative way to count tickets is to have a small number of sizes of tickets (e.g. small/medium/large). This is similar to estimating in story points, but it’s less granular. You still count tickets, but each ticket has a point value (for example, small = 1, medium = 2, large = 3).
- Works well if you just work the tickets as they come to you and you don’t have to do long term planning (e.g. handling production support, ongoing maintenance).
- Works well if a good portion of your work is not known up front and just comes to you when it needs to be done.
- Won’t work if you’re working on a project where management needs to know when it will be done.
More on story points
I often hear complaints from people that like story points. They say, “Management keeps trying to translate story points to hours!” My response to that is, what’s wrong with that? If you’re on a story point project, eventually you’ll have enough data to show that a 1 point ticket typically takes X hours on average, and so on. Isn’t that a good thing? Aren’t you better off if you’re able to translate story points to hours? Who wouldn’t want to have that knowledge?
This doesn’t mean that you can’t continue to estimate in story points, but remember, the whole point of estimating is ultimately to tell management or the business when you are going to get work done. There is still value in story points (more meaningful velocity charts, easier to estimate in some cases, etc.), but at some point the translation to hours is probably going to be necessary.
Use your brain
I find that people are very passionate about this topic, and the most passionate ones either think that you should always estimate in story points or never estimate in story points. I don’t like to use words like “always” or “never” because you can always find a situation where it makes sense to do things a certain way.
This is where you come in. You can read all of the books and blog posts that you want, but ultimately you need to use common sense and find the estimation method that works best for your team. There’s no harm is trying something and switching to something else later. The important thing is to do more of what works and less of what doesn’t.
This post is a part of a series of posts about iteration management. If you want to start from the beginning, go here.
If you ask people what they think of estimates, they’ll probably tell you that “estimates are always wrong”. But at the same time, management is taking those “wrong” estimates and treating them as a commitment to get work done by a certain date.
In reality, I think both of those things are true. The problem is that they seem so contradictory to each other, and this is where many teams get into trouble.
Many of you probably feel that estimating is a waste of time. It doesn’t make sense that you have to spend time each week estimating tasks when you could be using those hours to get things done.
I would love to live in a world where I didn’t ever have to estimate anything and all I would have to worry about is what I’m working on now and maybe what I have to work on next. Some of you might work in that kind of environment (maybe you’re in a small shop). But most businesses need to know how much things are going to cost, if Team A can complete a project before a certain date because Team B is going to need the functionality in place, if it’s worth it to build custom software or buy something off the shelf, or which consulting company to choose for a project. That’s the nature of smart business.
Or think of it this way – if you were going to build an addition on your house, would you not get several estimates from multiple builders and use those estimates as a major factor in who you picked to do the work? You certainly wouldn’t pick someone who said, “I do good work, so I’ll do the best that I can and then I’ll let you know how much it costs in the end.”
The fact is that if we want to be professionals, we need to be able to give good estimates, and we need to accept the fact that management is really asking for a commitment when they ask for an estimate. The whole reason that we estimate, regardless of the methods we use to do it, is so that we can tell management or the business when are going to get work done. Fortunately there are some things that we can do to come up with better estimates.
It’s much easier to estimate smaller things than larger things. What’s easier to estimate – the distance between your house and the one 3 houses down or the distance between New York and Chicago? You probably wouldn’t get either exactly right, but the margin of error will probably be less on the smaller distance.
The same is true when we’re breaking down a piece of work that we want to estimate. I like to break things down into the smallest unit that makes sense. Now that’s a pretty vague statement, but it really does depend on the situation. Ideally I like to break things down into the smallest testable unit that makes sense. That means that the feature provides some amount of value to someone (the business, IT, the dev team, etc.) and it can be tested and moved to production (or ready to go to production). There are always exceptions though – sometimes the smallest testable unit I can get to is still a week of work for 2 people, so then I’ll break the development down into smaller pieces (e.g. UI layer, business layer code, write some stored proc that is going to take awhile, set up an environment, etc.). Those technical tickets won’t be testable by themselves, but I’ll also have an overarching ticket that represents the testable unit (which is made up of the smaller techincal tickets). This at least lets us estimate smaller things so that we can be more accurate and have a better sense of how the work is progressing.
I’m not going to tell you what the maximum size of a ticket should ideally be, because every project is different. I’ve been on some projects where most tickets are under 4 hours of dev work, and I’ve been on other projects where the estimates were in days or weeks. The goal is to get the functionality broken down into the smallest chunk that makes sense for you so that you can get the estimate to be as accurate as possible.
Story Points or Hours?
There was enough here for it’s own post.
Categorization of work
On our team, we estimate analysis, development, and testing. So what counts as what? How does you categorize time spent by a developer writing automated tests? When a QA person writes a test plan, is that analysis or testing? What about meetings where you’re talking about a feature?
You can come up with your own rules of thumb, but here are mine:
- Time spent doing business analysis (this one is obvious)
- Technical design
- Research into how something works, whether this is business-focused or technical
- Time spent by team members in meetings discussing the feature (remember, 5 people in a 1 hour meeting = 5 hours)
- Coding, writing code, thinking about code, talking about the code, etc.
- Writing automated tests (if your QA people are writing end-to-end automated tests without developer involvement, I might categorize time doing that as QA)
- Manual testing
- Test planning
- Writing up bugs for a feature
- Writing end-to-end tests (see my comment in the Development section)
Other stuff (I might want to track the amount of time spent on these things, but not at a feature level)
- Team meetings not related to specific features (retrospectives, standups, iteration planning, other team meetings)
- Project management
How your team categorizes specific things doesn’t really matter, what’s important is that everyone is consistent and does it the same way so that you can get good data. You might want to check with your management as well because certain activities are capitalizable (like development, technical design, and testing) and others are not (like analysis).
The people doing the work should be the ones doing the estimating
This sounds pretty straightforward, but it’s really important. Let’s say that you know a lot about your application and you can complete feature X in 4 hours. But someone else who hasn’t been on your team as long or maybe isn’t as fast as you might take 16 hours. It’s not fair for you to give the 4 hour estimate and then hand it over to the person who is going to complete it in 16 hours. Maybe in that case you let the person re-estimate it at 16 hours, but that could throw your planning off. (This is another good thing about story points – when you estimate the size of a feature relative to other features, both developers might give the same estimate even though one will take 4 times the hours to complete it.)
I’ve been on some projects where we estimated a good portion of the work up front (because we had to bid on the project). That’s fine, but we never re-estimated anything. That led to lots of situations where a developer got assigned a ticket that they didn’t estimate, and once they had been working the task for as many hours as were estimated and they weren’t done, they started freaking out because they felt like they were behind, and this is when the sloppy code practices start happening. This happened because the people doing the work didn’t get a chance to give an estimate. If I give an estimate and I don’t finish it in time, then that’s on me, and I know it, so I don’t have any room to complain.
Nothing is more demoralizing than having someone else commit you to an estimate that you don’t feel is realistic. This just makes you feel angry, unmotivated, and unempowered. But if I make an estimate, then I’m a lot more likely to want to meet the estimate, even if it turns out that it’s going to take longer than I originally predicted.
Anything can be broken down
Any amount of work, no matter how complex, can be broken down into small, manageable tasks. The most complicated project is nothing more than a collection of simple tasks. The trick, of course, is identifying those simple tasks. But that scary project will become a lot less scary as you identify small units of work that need to be done. This exercise will also give you a better idea of all of the things you’re going to have to do, and you will be able to make better estimates on the small tasks.
This also applies to more nebulous roles like project management. Project managers are typically really busy and have to juggle numerous things at one time, but like anyone else, all of their work can be broken down, prioritized, tracked, and estimated. This will also free your mind because you have a system for keeping track of your work other than trying to remember everything in your mind (which is exhausting).
When to do the estimating
When the estimation happens depends on your team, your project, and how your team prefers to do it. I have a couple tips though.
Do the estimation as late as possible. It usually helps to have the work for the next iteration estimated so that you can do the capacity planning for the next iteration. You might have to estimate it earlier (so that you can plan out a release or bid on a project), but you might want to re-estimate that work when it gets closer to when you’re going to work on it in case the estimates have changed so that you can plan accurately.
Don’t have long estimation meetings. Some teams will have these meetings that last several hours long where the entire team goes through tickets and estimates them. I guarantee you that this will become everyone’s least favorite part of the week. Sometimes you might want to get everyone’s input on certain features, but for most things, you know who is going to do the work, so I would just let that person give their estimate so that you don’t have to tie up the whole team in mind-numbing meetings.
Remember the goal
Remember that however you estimate, the purpose is so that we can tell management or the business when are going to get work done. Use your intuition and come up with the best way that your team can estimate in order to meet this goal.
Read the next post in this series, Comparing estimation methods.
This post is a part of a series of posts about iteration management. If you want to start from the beginning, go here.
As an iteration manager, it’s your responsibility to make sure that everyone on the team has what they need in order to do their job and complete the work assigned to the current iteration. This can be tricky, but I find that it’s the fun part of the job.
Get people talking
Once of the biggest hindrances to success is a lack of communication. I’m sure you can think of examples where a lack of communication hurt the team. Maybe a developer reinvented the wheel because he didn’t know about the existing way to do it. Maybe your testers had issues testing some functionality because they didn’t know how to use the app correctly when a developer could’ve quickly showed them exactly how to do it. Maybe the users were confused or uncertain about some of the changes that you were making because they had questions that needed answers.
Someone needs to make sure that people are talking. This doesn’t mean that you have to do all the talking, you just have to make sure that you get the right people communicating so that everyone is on the same page.
An easy way to do this is to get the team members to sit together. This sounds obvious and many teams have at least part of the team sitting together, but it really helps having the BAs, developers, and QAs in the same area. Yeah, we might have email or instant messaging or phones, but people communicate so much better when their working in the same area. Plus, it increases team camaraderie, which helps communication even more (and it’s fun).
Be the unblocker
It’s your job to be the unblocker. This doesn’t mean that you just remove blockers for people, you also need to be able to see when people might be stuck and help them to get unstuck again. What complicates this is that many people don’t want to ask for help. They might think that they should be able to figure it out on their own, or maybe they don’t want to inconvenience people.
Whatever the excuse, there are usually some telltale signs that you can spot. Maybe you don’t see progress on something that should be moving along, or maybe you hear people sounding frustrated and complaining that they’re having trouble with something. This is where your intuition comes in.
Your job is to do whatever you can to allow people to focus on the work that they need to get done, which is what people tend to want to do anyway. But sometimes someone needs to see that there’s a problem and get the right people in the room to talk about it.
I’ve been on projects that have really well defined requirements and many others that didn’t, and let me tell you, it makes a huge difference. If your requirements are only 80% complete, then your developers are probably going to get it 80% right. When QA goes to test, they have questions about the requirements just like the developers did, and they either write up bugs or get in arguments with developers about how things should be.
When the requirements are well defined, you don’t have as many of these problems. In fact, you probably shouldn’t start working on a feature until you’ve all talked through it and make sure that everyone agrees on the requirements (which include the acceptance criteria).
What counts as “sufficient requirements”? That’s up to you. There are some cases where I feel like I need to write part of the code so that I can find all the things that I’m not thinking of. If I do that, at some point I’ll have a better understanding of what the requirements should be, and then we can meet about it.
What I don’t want are people not knowing how something should work, or people feeling like they need to make assumptions or decisions about how things should work, or people having to wait for requirements clarification. When things are well defined, everything tends to go smoothly because people don’t have to worry about not having all of the requirements and they can just work.
Keep things flowing
You can only move as fast as the slowest part of your process. Here’s how this often happens: management wants to get things done faster, so they come to you and say, “Let’s get more developers!” That’s great, but if you don’t have someone to write requirements for that developer or help test their work, things aren’t going to move any faster (or maybe they will at the expense of quality).
If even you aren’t in that situation, it’s virtually impossible to always have perfect balance of analysts, developers, and testers, especially on smaller teams. What do you do if you have 1 analyst, 3 developers, and 2 testers, but suddenly you really need 2.4 testers? The only way to get 0.4 of a tester is to have someone else on the team that is not in tester role help out with testing. This is where it’s extremely valuable to have people that can do several different roles well so that you can eliminate bottlenecks in the process and keep things flowing.
This especially applies to developers, because often times the bottlenecks happen when developers get work done faster that QA can test it (and many teams have a shortage of testers anyway). I like to encourage developers to learn how to do other tasks, like analysis and testing. This can help eliminate bottlenecks, and it makes developers better contributors in general. In some smaller teams, the developer is also the analyst and the tester and needs to know how to do these jobs.
Finding the bottlenecks
How do you find the bottlenecks in the process? This is exactly why we have big visible boards for tracking work items instead of just using online tools! It also helps if you break your tickets down into smaller tickets so that you can actually know when things get stuck. If all of your work items take a week long and three days in your developer doesn’t really have much to show for it, are they really behind or do they just need to hook everything up so that you can see it work? You don’t really have a good way to tell (other than asking and hoping that the developer can accurately and honestly give you the right answer).
We also want to find waste in the process. Usually this comes in the form of waiting. For example, waiting for answers from the business on requirements questions, waiting for testing environments to get fixed so that testing can start, waiting for management to make decisions, or any number of other problems. It’s impossible to eliminate all of these, but we can try to reduce them by communicating and finding creative ways to make things move better.
Think of the end of the iteration like a release date. Maybe you will actually be releasing to production. Either way, the goal is that functionality that goes into the iteration gets done. And by done, I mean all the way done. That means, that it’s developed, tested, and the users want to sign off on it. If you have another definition of done, you might be lying! When management hears “done”, they think that you don’t have to do any more work on it. So if you still have more work to do, that’s time that they aren’t accounting for in their mind, so now you’re in a tough spot.
As you get towards the end of the iteration, you need to make sure that everything that is going to be developed this iteration also can be tested by the end of the iteration. If you want to get a feature done by the end of the iteration, you can’t have developers checking in changes on the last day of the iteration if QA needs 2 days to test it because they won’t have enough time. On our two week iterations, usually we are cutting off checkins a day or two before the iteration ends, depending on how QA is doing. Developers might even help test if QA gets behind and can’t get it all done. Typically that doesn’t happen, so developers can keep working, but maybe they don’t check their code in until the iteration is over (or check it into a branch). This ensures that the QA team has enough time to test everything that is going into the iteration.
Every project is different, and every project will have it’s own unique set of problems, many of which you have never encountered before. At this point, your intuition needs to take over. Don’t underestimate the value of intuition and common sense! You can read all the books and blog posts that you want, and many of those will have good ideas, but it’s up to you to apply the ideas and tailor them for your situation.
One thing that you’ll find on pretty much every Agile project is some kind of card wall where features are tracked.
When I started working on Agile projects several years ago, I didn’t quite understand the purpose of the card wall. I thought, wouldn’t you rather just track everything in some online tool?
While tracking things in an online tool is still a good idea, the Agile card wall gives you a level of transparency that it’s difficult to get from an online tool. With a good card wall, you can easily see what everyone is working on, what still needs to be worked on, how long the remaining work might take, and what’s blocking your progress. There is so much to keep track of on a software project, and it really helps when you have a visual aid that can help keep everything in front of you so that you don’t have to juggle everything in your mind (or some online tool).
The trick is to tweak your card wall to give you the most transparency as possible. Here are some things that I’ve found help me get the most of my card wall.
Everything is a task
Everyone puts development tasks on their card wall. However, not everything that your team does could be categorized as a deliverable feature. In fact, there are lots of other things that you might have to do:
- Write up documentation on a wiki
- Onboard a new team member
- Set up a CI build
- Get ready for a demo
These are all tasks that are completed by development team members that take a measurable amount of time. So why not create a card and put them on the board? Maybe you use a different color card to indicate that it’s an internal task. This gives you two benefits — you know what people are working on and you can use the estimate to gauge how much work is left to be done.
Make card walls for your entire team
If our development card wall can help us track what developers are working on and how much work is left, why not create card walls for the entire team? You could have a card wall for BA requirements gathering, QA creating test plans, project management tasks, or some mini-project that you want to track separately. You get the same benefits that you get with a development card wall — you know what people are working on, you can easily see the status of the effort, you can prioritize the backlog, and it can help you get an idea of how much work is remaining.
Make problems obvious
Your card wall should alert you to any problems and blockers as soon as possible, so that you can remove any blockers and constraints that could keep your team from being able to do their job. Some ways that you can do this:
- Create a separate section of your card wall for blocked items
- If something is blocking a feature, put a post-it note on the card and write what the blocker is
- Structure your card wall so that you can see problems immediately just by glancing at it
Let’s look at some examples of some simple card walls and we’ll interpret what we see.
In this example, we can easily see that we are either developing things faster than we can test them, or things are getting stuck in testing due to bugs. We may need to assign more people to testing or work on reducing bugs.
There are a lot of features that are blocked. We should try and address the blocking issues and look and see if there is some greater problem that is causing things to get blocked. Maybe we need to write better features, or maybe we need to get more time from people in the business who can answer questions.
We don’t have much in our backlog, so the developers are going to be out of stuff to do really soon. We may need to get more people working on requirements gathering.
Things are flowing pretty smoothly. We have a good sized backlog, no blockers, the work is evenly distributed, and we’re getting a lot done.
The key is that we want to see problems as soon as possible so that we can take the necessary steps to correct the problem and keep things moving.
Address technical debt as part of the process
As much as we hate to say it, sometimes it makes sense to shove in lower quality, untested code in order to meet a deadline. I don’t like having to do this at all because of the risks, but when this happens, keep track of all technical debt that will need to be fixed and create separate cards for fixing it. Make these cards a different color so that they stick out. Then use your board to explain to your manager or project sponsor that you’re going to need some time to go back and fix the technical debt. Many managers want to pretend that the work is done once you release and will convince themselves that they can just move on without addressing the technical debt. You know better, and it’s your job to state that case and show why it’s important.
Refactor your card wall
Your card wall is yours. You create it, you manage it, and it belongs to you. Don’t be afraid to rearrange and refactor your card wall so that it can better meet your needs. Don’t make excuses like “we’ve always done it this way” or “we read a book that said to do it this way” or things like that. Your process and tools need to serve you not the other way around.
Read the next post in this series, Managing the iteration.
OK, if we’re going to talk about something called “iteration management”, we first should discuss what an “iteration” is. Turns out there are lots of different ways to deal with iterations, so I’ll go over some of the common ones.
Time boxed iterations (the Scrum way)
When most people think of an iteration, they think of a set period of time in which you perform a series of activities and then you do it all again in the next iteration. This is what you usually find in Scrum-based agile environments (doesn’t mean that you have to do textbook Scrum). In a Scrum-based environment, an iteration usually goes something like this (more or less):
1a. Capacity planning – determine how much work we can get done in the iteration
1b. Sprint planning – meet with the business and the team to figure out what to work on, maybe estimate tickets
2. Do the work
4. Maybe release to production, or at least say that a set of features are complete
5. Retrospective – team meets and talks about what went well, what didn’t, what to change, etc.
The length of an iteration is up to you. I’ve seen people do anything from one week to a month. The shorter the iteration, the less time you’ll have between production releases (if you’re releasing to production each iteration). With longer iterations, you often fall into the trap of doing a bunch of work and then trying to test it all at the end, which ends up being a testing nightmare (now you’re Agile team is just doing mini-waterfalls).
My favorite is a two week iteration. Sure, it would be nice to release to production every week, but I’ve found that the business likes the two week cadence of the process, and they get used to being involved at certain points in the process every two weeks. It also gives you more time to get the work done instead of having more meetings dictated by the process (since you only have those meetings every two weeks).
In the iteration-based model, your metrics are usually measured by the iteration. (For example, how much work can we get done in each iteration?) This model works well when you have a project that you need to complete by a certain time. You need to know if you’re on track, if you need to devote more (or less) people to project to get it done on time, you need to estimate work to know how long it’s going to take, etc.
Continuous flow (the Kanban way)
In this model, you don’t have time-boxed iterations. You just have a backlog where work comes to the development team at any time and they just work on whatever is most important. You might release to production on a regular schedule, or you might release whenever you feel that you have something significant to release. Some teams might even release to production multiple times a day.
The goal in the iteration-less model is not to see how much work you can get done in a certain amount of time. The goal is to see how quickly you can move a ticket through the process (we call this “cycle time”). The idea is that we want to minimize the time between when a business person has an idea and it becomes a reality.
This model works really well for teams that are doing maintenance work or production support (i.e. you aren’t working on a project where people need to know when the project will be done). Things like capacity planning and estimation start to feel like cruft in a situation like this, and it’s much simpler to just bring work to the team and do whatever is most important. You can still have regularly scheduled activities (e.g. demos every 2 weeks), but the process is much more fluid and free-flowing.
Which is best for you?
Only you can answer that! There are no rules on how to do this. I encourage you to think outside the box and come up with whatever works best for you. Regardless of the method, here are some things that I think are important that you should consider:
- How can I most quickly diagnose problems?
- What works best for the stakeholders, management, and the business?
- What allows us to best handle (and even encourage) changes in priorities?
Most of the rest of this series is going to focus more on the time-boxed iteration method of developing software. Many of the ideas will still apply to the flow-based approach, but the time-boxed method is more involved and has a lot of interesting nuances that are worth talking about.
Read the next post in this series, Card walls.
If you haven’t heard of iteration management before, maybe you hear the word “management” and you think “project management.” It’s similar, but not really. Here’s how I distinguish the two – project managers manage projects across iterations and maybe across multiple teams. An iteration manager manages the work done by one team within an iteration, although there might be some coordination with other teams.
An iteration manager helps to optimize the efficiency of the team. People on software development teams tend to like to focus on their tasks. They will strive to limit distractions and do whatever they can do get their work done, and sometimes this even comes at the expense of the team. In a way, this is good, because if you give someone a task to do, you want them to get it done efficiently. But in order for this team to truly function like a team, we need someone whose head is not in the weeds who can look at the bigger picture and make sure that the team is working in the most optimal way. This is where you come in.
An iteration manager is an unblocker. If team members are blocked by something, the iteration manager should try and get it resolved for them. An iteration manager should be proactively searching for blockers before people raise an issue because some people will either spend too much time trying to figure things out on their own or just not tell anyone that they are blocked.
You might have to be creative with how you do this, but you will learn to see the signs of someone being blocked. If you have a board/wall/online site/etc. where you have the iteration’s tickets, watch for things piling up in a certain status (whether it’s analysis, development, or testing). If you have people on the team that are working on multiple tasks at once, it could be because they can’t finish the first task until they get an answer from someone, so they’ve moved onto the second task. Or maybe you just notice that a certain feature isn’t getting done as fast as you thought it should and it causes you to ask a question.
Overall, your task is to help the team work as efficiently as possible. This doesn’t just mean “fast” (although that’s certainly part of it), it also means that you ensure that software is going to meet the needs of the business and the people that use the software. In order to do this, we’re going to use everything at our disposal, including data analysis, communication, and lots of intuition.
What I love about iteration management is that it allows you come up with creative ways to achieve the goal, and there’s no one right way to do it. The rest of this series is going to give you a bunch of tools, tips, and tricks to get the job done, but ultimately you’ll have to use your own intuition to do it well.
Read the next post in this series, What’s an iteration anyway?
Iteration management is the art of managing the estimating, planning, and execution of a software development project over a short period of time, whether this is done individually or in a team environment. I call this activity an art, because while there certainly is a scientific element to it, it requires excellent communication skills and a strong intuition. You need to understand what is realistic for your team while getting things done.
For some of you, this might be a new career path that you didn’t know about. Many people assume the next step from being a developer, tester, or analyst is to move into some sort of traditional management. Iteration management allows you to stay closer to the team and probably continue doing the work that you’ve been doing while providing value in a new way.
This topic is way too big for one topic, so I’m splitting it up into a series of posts:
- What is iteration management?
- What’s an iteration anyway?
- Card walls
- Managing the iteration
- Comparing estimation methods
- Data analysis
- Personal iteration planning
- Burndown charts
- Capacity planning
- Involving stakeholders
- Managing the backlog
- Working with management
- Keeping up
I found this post that I wrote in 2010 and the 2014 version of myself can’t say it much better, so I’m going to regurgitate the same post for your reading enjoyment.
Go to a software conference near you and you will probably hear talks on new languages, new frameworks, how to design code better, how to test better, and the like. What no one ever seems to talk about is how you can go faster.
We need to get faster at what we do, because it just takes too long. Why do you think companies and managers are always pressuring people to get things done faster? This is a tricky situation though — the challenge is to find ways to get faster without cutting corners, ignoring unit testing, and writing poorly designed code. Anyone can get things done faster by deciding to not write tests or not think about how to design code well, but that’s cheating. That just leads to more bugs and more work down the road.
Don’t ignore the importance of getting things done in a timely manner. I got to be a project manager on a project once, and every day I was looking at our feature wall to see how we were doing and if we were on schedule. I guarantee that your manager or project manager is doing the same thing. That experience and awareness helps keep me from not wasting time adding non-essential features or spending too much time over-analyzing things (be especially careful of this one if you’re pairing). I try to work with a sense of urgency, as if I’m competing with someone else who is trying to complete the same project faster than I can (without cutting corners).
One way to get faster at something is to practice. Find a simple coding problem, like the ones mentioned here. Pick an easy one (nothing with crazy math algorithms, complicated recursion, etc.), or maybe take a complicated one and simplify it. Try and find something that you can do in 15 minutes or less, but still involves writing tests and creating multiple files.
The first time through, go through it like you normally would. Think about design, write good tests, and make sure you understand all the nuances. Then go back and do it again and time yourself. Do it over and over, and try to beat your previous time. At this point, you’ll know how to design the code pretty well, and in order to get faster you’ll have get faster at moving around your IDE, find ways to generate code, or learn new tricks. This is what you’re trying to learn. Remember, you’re not allowed to cut corners! You still have to write tests and write well-designed, readable, clean code. Check your code each time you’re done and make sure you didn’t compromise in these areas.
In sports, there are the teams that are good on paper and there are those who execute during crunch time. You may be good at writing well-designed, well-tested code, but if you can’t get it done on time, no one is going to care. So take time to hone your skills and become a faster developer.
I tweeted a quote today that I found to be very insightful.
A poignant reminder of why I'm here. pic.twitter.com/H9Nkbnm4Qt
— Jon Kruger (@JonKruger) April 8, 2014
You could take this two different ways. I take it to mean that I need to just the right amount of testing in order to be confident that my code works and will continue to work.
You could also see it another way, as pointed out by some others:
@JonKruger I think the line which follows that quote is also important; as an industry, that level of confidence might not bee too high.
— Steven Harman (@stevenharman) April 8, 2014
— SeattleFan4Dan4 (@TJWilk_WA) April 8, 2014
There are those out there that think that testing is a waste of time and I could see some of them taking a quote like this and saying, “I’m a good enough developer that I don’t need tests in order to write code that works!” (I heard someone say those exact words once, sadly.)
@JonKruger Most developers do the same thing… they just have way too much confidence in their coding skills. ;)
— jared richardson (@jaredrichardson) April 8, 2014
The discrepancy is due to what the word “confidence” means in this quote. If it’s just confidence that it works now, then you might be fine without tests, but if it’s confidence that your code will still be working a year from now after someone else has had to modify it, then you probably want tests. (Heck, I want tests so that I can change my own code tomorrow!) I find that the problem is that on many teams, developers define confidence as “good enough that QA can look at it” and then achieving true “confidence” is someone else’s problem.
Maybe we should change the pronouns.
We (the team) get paid for code that works, so our philosophy is to tests as little as possible to reach a given level of confidence.
Now this changes things. What’s the best way to test as little as possible? Automate all the tests that should be automated. Let QA spend time doing valuable things like exploratory testing instead of doing loads of manual regression tests. Funny how you can turn everything on it’s head when you take what is perceived to be an individual’s problem and make it everyone’s problem!
When we write automated tests, we want to do it in the simplest way possible while achieving the highest level of quality for the time we invest in our tests. Unfortunately, this is easier said than done.
Writing tests can be hard. It’s hard figuring out what kinds of automated tests to write. We can write unit tests, integration tests, functional/acceptance tests, performance tests, or just do manual testing. We have to decide if developers are writing tests or if QA people are doing it (or both). It’s really complicated!
Let’s look at some ways that people sometimes tackle the testing problem and then discuss some different ways we can approach it.
The black box
On some projects, the team is testing a big black box. Usually this is where QA people are doing all the testing and developers aren’t writing automated tests. If you read my blog at all you know that I do not like this approach (unless time to market is really that important) because it leads to lots of bugs and makes refactoring anything really scary. In this scenario, QA testers typically only control the inputs and outputs into the black box (which is usually a user interface of some kind). This leads to problems like having to jump through a bunch of hoops just to test something that’s farther down the line of some complicated workflow.
If you’re a fan of test-driven development, you will write a lot of unit tests. In this case, you’re slicing your application up into very tiny, testable units (like at a class level) and mocking and stubbing the dependencies of each class. This gives you faster, less brittle tests. You still probably have QA people testing the black box as well, so now we’re using two different approaches, with is good.
I’m not satisfied
While each of the previous two approaches have their positives, they both have their negatives. I’ve already talked about the negatives of all manual QA black box testing, so I won’t go into that again (although I’m always up for a good rant). But writing lots of unit tests has its problems as well. For example:
- Tests with lots of mocks and stubs, and failing tests that fail because someone refactored a class any now my stubs and mocks are all hosed. You know the feeling.
- Testing an individual class or method is great, but my tests aren’t always testing something that has business value on its own.
You own your application. Your application does not own you (at least it shouldn’t). We also own the testing of our application and should be able to test it in any way that we can think of. The goal is high quality, low cost of maintaining the test suite, and speed to market. How can we best achieve this goal? (And don’t just give me the first textbook answer that pops in your head.)
There is no one-size-fits-all method for testing applications, and there isn’t even one single best way to test a single application. So what if we used many different approaches and broke our system up into chunks so that we can use each testing method to its fullest potential?
In this case, I’m dividing up my application into separate modules, each with its own purpose, function, and business value. Some may have a user interface component and other might just do some task behind the scenes. I can decide how to best test each module individually. Maybe some modules are done with all black-box acceptance testing, and other modules are done with lots of unit tests. Even within the black-box modules, I might still write unit tests. Of course, I’m still going to have some end-to-end tests (manual and/or automated) that test the whole system working together, but I don’t have to test the majority of the functionality this way.
My favorite kinds of tests are ones that test a system working together because I can specify the inputs and outputs and I don’t have to deal with tons of stubs and mocks. Now if you try to test the whole application end to end this way, it can be a bit cumbersome. But if you have a smaller module that you can test end-to-end, now you can have clean, readable, well-defined tests that don’t have tons of mocks, and the tests define some business function that the module needs to perform. My module might still work independent from the UI or the database, so I might still be able to stub those out and have fast tests. This feels like the kinds of tests I’ve always wanted – tests that test a module end-to-end but are able to run fast because I can still isolate dependencies.
Hey, look, it turns out that modular applications is a good idea in general! It’s way easier to deal with lots of smaller applications that work together than dealing with one monolithic application. Those of you with large solution files and long compilation time (I’m raising my hand) know the pain of dealing with large applications.
The emerging blob
We like to talk about “emergent design” and that we can write tests first and let that drive the design of your code. That is true, but your codebase will evolve into a monolithic (albeit well-tested) blob of an application that assimilates all code into it’s large collective.
The only way you’re going to have a modular application is if you draw the lines in the sand up front. This can be really hard to do when you have a newer application and you don’t have a ton of insight to tell you how to keep things separate. Compounding the confusion is the fact that you might have a single database for the application as a whole, which I think is fine. You can multiple modules that use the same database, even the same tables. Sure, it would be better if you can keep the tables in separate databases, but sometimes that’s not possible or realistic.
You might start out with certain modules and then realize that you created a separation that is too painful to maintain. That’s OK, it’s much easier to combine two modules than it is it try and separate things into modules after the fact!
Once you’ve defined your modules, now you can decide how to test them (QA and devs should work together on this!).
This feels better
- Cleaner tests with fewer mocks that test mini-systems that provide some function or business value
- More modularity means I can change code without potentially breaking as many things
- Smaller solution files!
I really like how this sounds.
Older Posts »