Yet, as with anything in life, there were challenges and room for improvement. I remember attending a noteworthy backlog maintenance meeting with one of the teams. This particular team was incredibly strong, so I simply attended to check how well they were grooming. To be honest, I was hoping to share some lessons from their approaches with some of the less experienced teams. Jon was one of the lead engineers on the team. He had been a Scrum Master for a long while, so his agile chops were mature and balanced. However, I was surprised when the following happened:
Max, the Product Owner introduced a User Story for the second time in maintenance. The team had already seen it once and had realized two things:
1. It was bigger than a Sprint’s worth of work for the team (call it an Epic or non-executable Story), and
2. They needed more information about the legacy code base surrounding implementing the story.
So they created a Research Spike that represented technical investigation into the story. This session was the first time the team had got back together after their learning from the Spike. Jon had taken the lead on the Spike, working with two other team members.
He went over the implications from a legacy code base perspective. Jon started the discussion. He and his small team recommended they split the Epic into three sprint-digestible chunks for execution. Two of them had a dependency, so they needed to be worked in the same Sprint. The other needed to be worked in the subsequent Sprint in order to complete the original Epic.
Jon and his team had reviewed the legacy code base and said, in order to do the work properly, it would take a total of approximately 40 Story Points. However, he pointed out this might be perceived as excessive and approximately 25 of those points would be spent on refactoring the older code base. The specific breakdown was 18 points for the new functionality and 25 points to refactor related legacy code.
The Product Owner excitedly opted for the 18 points and deferring the refactoring bits. Jon and his small Spike team wholeheartedly agreed and the entire team went along for the ride. From a backlog perspective, the 18 points worth of stories became high priority and the refactoring work dropped to near the bottom of the list.
And the meeting ended with everyone being happy with the results.
I decided not to say anything, but I left the room absolutely deflated with this decision. It was opposed to everything we had been championing at an agile leadership level. Clearly put, we wanted the teams doing solid, high quality work they could be proud of. In fact, all of our Definition-of-Done and Release Criteria surrounded those notions.
If the cost of this Epic was approximately 40 points to do it right, then that was the cost… period. Splitting into the parts you agreed with and the ones you didn’t agree with, were not really options. Sure, each team needed to make the case to the Product Owner surrounding the why behind it, but it was not a product-level decision. It was a team-based, quality-first decision. De-coupling the two broke our quality rules and that decision would haunt us later as technical debt.
To close this story, I used my not-so-inconsequential influencing capabilities to change this outcome. We decided this Epic was important enough to do properly and the approximately 40-point cost was worth the customer value. In other words, we made a congruent and sound business decision without cutting corners. The team fully appreciated this opportunity, without second-guessing and guilt, to deliver a fully complete feature that included the requisite refactoring to make it whole.
Now, I only hope they continue to handle refactoring opportunities the same way.
Watch our free webinar, A Tester’s Guide to Collaborating with Product Owners. Apply password, “ZenBox284” to watch.
I would like to make the coupling even stronger between technical debt and refactoring. To me, you refactor away technical debt. You identify the debt and the effort to remove it is refactoring. Code is a primary place for it, but I believe you can and should refactor other items, for example:
Clearly I lean towards a broad-brush view to refactoring responsibilities and connecting them to the various kinds of technical debt. From my perspective, I’d recommend you deal with it as broadly as possible within your own contexts.
While I was coaching at the email marketing SaaS firm, a trigger word in our team discussions was “hack.” Whenever a team member spoke about hacking something together, we knew it would create technical debt and need refactoring lator. So we worked incredibly hard to avoid “hacks.”
Once you show the discipline to hold the line on new work, you can go back and start refactoring legacy crud that has developed over time. This usually is a longer-term strategy in many organizations and requires great persistence. It is also a moving target to some degree, so patience is needed as well.
I like to engage the team in identifying refactoring targets. Avoid the we-have-to-fix-everything syndrome and ask the team for the highest priority refactoring targets by way of value–for example, removing impediments to the development team’s efficiency or capacity.
Balance is a key in refactoring. Attack technical debt in all its forms and do not necessarily focus on one component or type of debt. You want to look at your entire codebase, tool-base, script-base, documentation-base, etc., in your retrospectives and select high-impact, high-return refactoring opportunities. Then apply a bit of relentlessness in pursuing and improving those areas.
Even though I talk about refactoring being an organizational and team responsibility, it does not get supported by magic. Teams need to identify (document) their refactoring work on their Product Backlogs. The business case for each improvement needs to be explored or explained, particularly if you are going to get your Product Owner to support you. So yes, it is a responsibility. But you need to put the rationale and the ROI in clear business terms. Then connect the dots back to the ROI after you have refactored the code, perhaps discussing or showing improved implementation speed in a Sprint Review.
Remember that refactoring often has a cost in time-to-market. Bugs take longer to fix or cluster in ways that influence customer confidence and usability. Maintainability is a strong factor in being truly nimble and creative. At the email marketing firm we often selected and justified our refactoring targets by how they would support our future product roadmap and support faster implementation times.
Then, when we had completed the refactoring, we would look back on those improvement estimates and speak to the reality of the improvements–connecting the dots, if you will.
One of the hardest things to do in many organizations, those with debt-rich legacy systems, is to prioritize the technical debt. There is so much and it is causing so much harm, the inclination is to try and fix it all at once. But nothing could be more detrimental from a business perspective. As you would handle anything on your backlog, prioritize it and systematically attack it.
I have often heard the notion that a value proposition of building solid test automation is that it provides a safety net so the team can courageously refactor. The point is, if there is little to no test automation, teams are reluctant-to-fearful to refactor because of side effects and how hard it is to detect (test for) them. I have found this to be incredibly true.
So a part of any refactoring effort should be focused on building test automation coverage. The two efforts strategically go hand-in-hand.
Most agile teams today have multiple tempos: sprint tempo, release or release train tempo, and calendar or external release tempo. You want to think about your various tempos and perhaps find opportunities within them for a focus on technical debt and refactoring. For example:
Many SaaS product companies have downtime periods in the calendar year when they do not necessarily want to release new code to their clients. At the email marketing firm, our time was over the Christmas holidays. From November to December each year we needed to keep releases to a minimum while our customers focused on holiday revenue. Given that, we would plan Refactoring Sprints & Releases over that period. Sometimes we focused on product code, for example, broad-brush defect repairs and component or feature refactoring. Another season we worked on our continuous deployment capabilities–focusing on scripting, tools, and deployment automation.
It was a great way for us to make investments and not disrupt our Product Roadmap plans.
And the final strategic point is making it clear to everyone that technical debt and refactoring are an ongoing challenge and investment. They will not go away on their own. Even if you are implementing a greenfield project, you will be surprised how quickly you gain refactoring debt. It is driven by the nature of software–the customer needs change, technologies evolve and change, and teams change. In other words, change happens, so simply factor it into your strategic plans, roadmaps, and ongoing product backlogs.
Subscribe here for our most recent blog posts, discounts on Zenergy trainings, workshops and more.
The sub-title for this article was: it’s not a choice, it’s a responsibility. I hope the introductory story helped to crystalize that point. But I would like to emphasize it even more now.
Stakeholders will rarely tell you where and when to refactor. In fact, they typically hate the notion of refactoring, infrastructural investment, ongoing maintenance, etc. Instead they usually push their teams toward more and more new features. This pressure is organizational and will seep into the product organization, each Product Owner, and their teams. However, just because we are under pressure, it does not mean we need to abdicate our responsibilities and blindly succumb to it.
Rather, we need to activate our craftsmanship, our professionalism, our responsibility for doing good work and our courage to deliver that work. In other words, delivering software that will stand the test of time, exceed our customers’ expectations, and we can be proud of. All of that might sound too simplistic, but it is a core part of the principles behind the Agile methods.
Beyond words, each agile team and organization needs to make its technical debt (risks) and its refactoring efforts (investments) transparent. They need to become part of the everyday discussion that teams, managers, and senior leaders have as they transform their organizations towards agile execution. Striking a transparent balance should be the goal. I strongly suspect that everyone’s common sense and gut feelings will let him or her know when they have achieved it.
Stay agile my friends!
Bob.
Up Next:
