Bugs, Bugs, and More Bugs!

In my agile coaching and training journey, I spend a lot of time discussing a wide variety of topics; for example, hardening sprints, refactoring and technical debt, etc. But certain themes form a Top 10 topics list everyone seems interested in. One of those items is how to handle bugs. I get questions like:

Do you estimate bugs (planning poker – points)?
Are bugs equivalent to stories?
When do you file a bug while sprinting?
Do you count bugs as part of your velocity?
Can you deliver a story in a sprint with bugs still open?

These questions surface in my classes and at conferences with amazing frequency. I often smile inside in amazement at the level of interest focused on bugs. But it is an important subject for agile teams, so here are my views toward handling bugs in the above contexts. Now remember, in agile transformations, context counts. Nothing I recommend should be blindly applied everywhere. My only intent is to provide you with some guidance to get you thinking about how to handle your bugs.

Do you Estimate Bugs (Planning Poker - Points)?

In this case I’m talking about planning poker and estimating or applying points to bugs. I’ve thought for years that you probably don’t want to estimate more common bugs. It can be quite wasteful. That is, by the time you estimate a fix you might as well actually fix the bug. But in backlog refinement, we tease apart estimation vs. repair time. So often you spend nearly as much time estimating the bug as you would spend fixing it in a later sprint. I prefer not estimating most bugs, but focusing effort toward investigation and then immediate repair. However, I do see two situations where it makes sense to estimate bugs. First, it probably makes sense to estimate the hard ones or those that are fairly complex or cross-cut major areas of your systems. In my experience, one category of these is performance bugs. They typically take lots of effort and time to resolve. I could easily see running planning poker to explore the nuance (and size) of these bugs. Second, I often recommend packaging bugs into themes rather than estimating individual bugs. Theme-based defects packaged into a user story might represent 10 individual bugs. In this case, they may all be UI bugs related to a specific workflow within an application. That way the total starts weighing in as a typical or average user story and the effort to estimate makes more sense.

Are Bugs Equivalent to Stories?

I remember a client a long time ago who was using Jira to track their defects prior to going agile. When they made the move, they obviously selected Jira as their agile ALM tool. This created a clear path for their 5000+ historical bugs to suddenly become stories in Jira. These were then augmented by real functional stories, so the product backlogs became huge. What they learned is bugs really aren’t equivalent to stories in size or in attention, as I mentioned above. Furthermore, they realized the large number of bugs diluted the focus and attention to the backlog. In other words, I don’t think all bugs should wind up being stories. Perhaps twenty percent of the largest and most complex should, but the remainder (small, cosmetic, historical, trivial, straight-forward) should not be. Again, aggregating them is a more effective strategy.

When Do you 'File a Bug' While Sprinting?

Never. You try to build software without bugs, and if you find them, you fix them immediately. But if this isn’t your reality, perhaps it’s a fair question. If you’ve ever taken one of my classes where I discuss Definition of Done, you know one of my favorite constraints for agile teams is no bugs per story. That is, if the team has created 15 defects during construction of a story, I’d expect all 15 to be resolved before gaining credit for the story, independent of the priority of the bugs. Now, I’ve been accused of being harsh with that constraint, but it does influence keeping your code base clean. I asked my colleague Shaun Bradshaw to review this article and he made the following comment: While I agree with this, wouldn’t it make sense to “file a bug” if the team isn’t co-located? Maybe I misunderstand what you mean by “file” a bug, but I’d think you’d want to file it to track it across multiple locations so it’s more easily retrieved, reviewed, discussed across the team. Shaun brings up an interesting exception case, leveraging defects as a communication mechanism during the sprint. I’m fine with that, as long as we’re not using it as the only communication mechanism. So what if a bug does slip out of the sprint? Or what if you find a legacy code bug during sprint work? Well, I think you file it. So consider the sprint boundary as the filing point. Try to focus on collaboration and repair during the sprint and if any bug cascades beyond the sprint, the normal action is to file it.

Want some new ideas and techniques for helping your Product Owner and team(s) to deliver better received and higher value products?

Watch our free webinar, A Tester’s Guide to Collaborating with Product Owners. Apply password, “ZenBox284” to watch.

Do you Count Bugs as Part of Your Velocity?

Again, it depends. If you size your bug stories as part of your backlog refinement, then yes, they are planned for a sprint and result in velocity. However, there is a potential dilemma. For example, what if you are building a story and find bugs related to your design and coding efforts? As you find those bugs, do you log them and consider them stories? Or are they simply related to the original (parent) story and repairing them is part of the original implementation effort estimate? I’ve encountered quite a few teams who handle in-the-sprint bugs as stories. They estimate them in points and plan them for later. I’ve had many discussions with senior leaders of teams who do this and struggle with their teams’ commitment to work estimation and delivery. Execs feel they’re paying for the story multiple times, at the original estimate and for each subsequent bug story. They’d rather the teams build the story right the first time. There probably is a proximity clause here as well. For example, let’s say a team delivers a story in sprint 3 and later, in sprint 7, they discover two bugs related to the original implementation of the story. What should they do? First, I’d say to fix them in that sprint. You’ve delivered much higher value software in sprint 3 and cleaning it up should be a priority. But what about the points? I’m torn. If the bugs can be repaired quickly, then I’d say do that and don’t worry about the points. If they require significant time, I’d say write a story and estimate/point it. I’d also make it visible/transparent you are cleaning up a previous bug and would allocate space for time to repair.

Can you Deliver a Story in a Sprint with bugs Still Open?

This is probably the easiest question to answer, but it still has two parts. First, it depends on your Definition of Done (DoD). If it precludes delivering stories with bugs in general or of specific types, then yes, you need to fix them before closing the story. If it doesn’t, then you can deliver and demo it. However, you should probably log the bugs for later. I often see teams make a priority distinction here. That is, in their DoD they’ll indicate fixing P0 – P2 bugs within the sprint and defer others to later. I think the important thing is to have some transparent rules around your handling and consistency across your teams.

Wrapping Up

Just as talking about bug handling is usually a hot topic in agile circles, everyone may not agree with my advice. It’s the nature of certain topics. Given that, I encourage readers to add comments with your experiences in handling bugs in agile contexts. What have you seen work? What doesn’t work? And importantly, does anyone have something to share about PREVENTION?

Stay Agile my friends! Bob.

Up Next: