The IT Manager's Dilemma with Software Debt
“The team continues to complain about working with that legacy codebase because it has so much software debt. That software debt slows them down in feature delivery and they are wondering if we can push for priority to be put into paying it back some?” asked the ScrumMaster. The IT Development Manager looked distraught about the request. She knew that paying back some of the software debt would be a valuable effort but what would the business say about the priorities? “Tell the team that we’ll start paying back some of the software debt starting in the next release. We must get the current release out the door and a change in priorities won’t allow us to get there.” the IT Development Manager said.
Considering the circumstances I cannot tell an IT manager that this approach is not the right way to go. In many cases the IT manager is not in a position to push for technical priorities even when they will provide value to the business. As teams continue to develop on a system without managing the software debt effectively feature delivery throughput decreases. The following picture shows a team’s feature delivery throughput versus time spent stabilizing each release over time:
There are many reasons for this software debt accrual:
- Pressure of the deadline
- Inexperienced team members
- Specialization
- Over-complication
- Bad design
- etc…
IT management has looked for ways to minimize the effects of software debt. We introduce processes that in theory will reduce software debt but in reality seem not to lessen the effects at all. Tools are introduced that will ease the software development process but we still see similar or potentially new mistakes made by team members. Individual team members are asked to specialize in particular software development disciplines such as Database Administrator (DBA), Quality Assurance (QA), or Business Analysis (BA). Although we do each of these specialized roles more efficiently it seems that the product delivered still is accruing software debt. So what do we do?
It is my experience that teams that adopt agile test and engineering practices within an organization that supports collaboration between business units and development teams are more successful in the containment of software debt. These teams tend to minimize software debt and will at the very least deliver with consistent throughput release after release. In some cases I have seen teams accelerate the velocity of their feature delivery throughput over time. The following figure represents the problem IT managers have in deciding to manage software debt effectively on existing legacy software:
A team will have to slow their current feature delivery significantly in order to get consistent throughput over time. I would suggest that managing the software debt effectively would be the best decision for a business relying on this software. Software is a valuable asset for businesses that can:
- Reduce costs for business processes by automating significant portions
- Provide information to business quickly so they can make better strategic decisions
- Attain market share providing shrink-wrapped software to meet the market’s needs
- Reduce system decay on existing software assets so they can be used into the future
- and much, much more…
Still, given all of these reasons it is difficult to take on software debt in the wild. We must understand that a typical IT manager has many influences on their decision making, as well:
- Business pressures for a set of features on a certain date for a set cost (The Iron Triangle)
- Expectations of feature delivery based on past software releases
- Unreasonable deadlines due to business issues such as lapsing support contracts and executive management promises
- Perception of their organization and how that reflects on their capabilities
- A compensation plan that does not reward managing software assets for future capability
- etc…
Given all of these circumstances I believe that IT managers are making the best decisions possible. How can we help IT management support our organizational software assets effectively and minimize the effects of software debt? What approaches will allow the software delivery teams to manage software debt while delivering essential features? How can business get more involved and increase understanding of this dilemma that will affect the organization’s capabilities over time? I am interested to hear from anybody who reads this. What are your suggestions?
It depends. I agree, in general, that IT managers are behaving in ways that seem rational for the system that are participating in - if by rational we mean that they are figuring out the things that get them rewarded and doing those things.
That is moral relativism; you can use the same logic to excuse the American Slave-Holding Southerner in 1860.
So while I may not like the behavior, and I might not even think it’s right, I admit that it’s extremely rare for an IT line manager to rise above it.
I don’t look to IT managers to solve this problem. It is not the IT manager that actually _DOES_ the shortcut of “bad” technical debt; he simply exhorts and begs and pleads the tech staff to go faster.
It is the tech staff that makes the hacks, and thus the tech staff that needs to change behavior.
How do they do that? It’s pretty simple. Give estimates that are _responsible_. When asked to compress, negotiate scope, not features. Constantly improve our craft. Periodically reflect on the work we are doing. Mentor others and seek mentors. Most impotantly, never, ever make the same moral mistake of the Nazi Prison Guard when taking on tech debt: “I was just following orders.”
I’m not saying put you company out of business because you need to take a “principled stand”, I’m saying technical folks need to take responsibility for our tech debt and not blame management(*).
Now, I don’t want to eliminate all tech debt. I don’t even think that is possible. But if we can reduce it by a sizable fraction - say cut the average (bad) tech debt of a shop in half - we will significant increase the velocity of software development, thus increasing the financial stability of our companies, and our own sense of health and well-being.
The void of bad code is a pretty big hole - and empty bucket. If what we do can be a sizable splash into that bucket, well, I would be pleased.
Regards,
-matt heusser
(*) - If you look carefully at my comments during “the weaker brother” at tech debt, that is one thing I consistently did - took personal responsibility for my tech debt choices, instead of blaming the management bull-whip. We need more of it.
(hmm. Long Comment; I’ll blog this too!)
I totally agree the responsibility for fixing the debt, or better yet not incurring in the first place, lies in the hands of those actually doing the work. The problem that I see with this is that the if person doing the work reports to the IT manager, then when you stand on your principles you’ll incur the wrath of that IT manager at review time, and even with a GREAT team they won’t get compensated correctly for the high value work they did. Effectively, you’ll teach the team to ignore the increase of debt because it PERSONALLY affects their paychecks instead of just violating their software morals!
My hypothesis is that the root cause for this is really twofold:
1 - The IT manager committing to the delivery of the release on behalf of the team.
Let’s see, backlog of 100 points, team’s past velocity of 10 points/sprint, 2 week sprints. OK, [insert CIO name here], we’ll be done int 20 weeks… I guarantee it. Oops, we’re a trending a bit late/overbudget/underscope, but I promised that to the CIO already, I better go bust some heads to get that product released. Now the IT manager is doing this with what they think to be good data: history. Ignoring the other bad practices (like velocity is a guarantee), let’s just assumet that this history was paid with some % of debt, and we haven’t recognized that this needs to be repaid. We are only paying interest on those “day 2″ items, and are estimating our ongoing budget with that same amount of debt. So the net result is that we are not paying off the debt, and we are increasing the balance. Thus, those interest payments are getting larger (as shown graphically by Chris above).
2 - The IT manager is not being measured correctly.
What happens if you were to lessen the hardness one side of the iron triangle: features. The IT manager has very little control over the number of features that the team can deliver… and this really is the responsibility of the team, in my opinion. What if instead we create a metric that compensated them on future value like we *should* do with CEOs… (how much did the mortgage lenders make again?) What this metric actually is probably will probably vary depending on the values you have for the product.* For example, if your OS is having severe security vulnerabilities, then perhaps the minimum # of updates pushed to clients over the next year is a good metric (remember BillG’s security speech?). However, if constant evolution of the product is required, perhaps the maximum # of updates pushed to clients over the next year is a good metric (how many releases is Rally up to this year?). The point is that the pressure being exerted on the manager needs to be the correct pressure or you could end up with the infamous “unintended consequences.”
Hmmm.. Depends what you call technical debt. For a _lot_ of people technical debt is just crap that you couldn’t be bothered to do before the software went out. Why? Because its too hard, required too much thinking or required too much refactoring to be able to effectively test.
Technical debt is very hard to explain to the “IT Manager” (of the clock watching sort) because they don’t really care what lies under the hood, and nor should they. (Whether these people should actually exist is another point completely).
However, software does indeed need to be delivered, so compromises must somehow be made between the unattainable perfection, and something that works well enough to serve the purpose. Remember that the purpose must also include the companies future profitability too, so making drastic compromises should be done only under drastic circumstances. Lots of teams do this quite effectively.
Sheer laziness on the other hand, is much more cancerous, and time and again I see “technical debt” built up simply because it was easier to not bother. Team Leaders (not managers), or teams themselves should constantly evaluate where this build up is coming from, and eject the lazy. If they can’t even be bothered to fix the small stuff, how can they be trusted with more macro changes.
Telling comments like “//fixme - its late now and I must get some sleeeep” are a sure sign of a destructive programmer within your team.
Sorry long comment - short version is: Technical Debt should be a planned, well thought out compromise. There is crap in your software too, and if that comes from laziness, then don’t be afraid to make the distinction.