Technical debt is a rabbit hole that goes deep, really deep…

I don’t care how brilliant a developer you are, every piece of code you will ever write will add to technical debt. Yes it will, and it will irritate you to no end.

White Rabbit

Technical debt is one topic that instantly sparks debate and ignites controversy. I see the talk of technical debt crop up at work (even though we are still in the earliest stages of the project and haven’t accumulated much), and I see the need to manage technical debt in the code I write myself (even if my projects relatively small in size).

I believe what makes technical debt a development pain point is the fact that there are literally endless ways to continue to improve, optimize, refactor, enhance and generally better the code. Since most capable developers are perfectionists, these opportunities for improvement are a persistent and nagging temptation. It’s an issue that can affect productivity, team morale, and that has direct implications for the business value you are trying to extract from your team’s work.

Given the importance of the topic, I thought I’d share my own perspective and experience managing technical debt in a constructive and healthy way that still makes business sense. If you are lazy, you can skip to the “definite DO’s” at the end of this article.

Technical debt is a rabbit hole that goes deep, really deep…

Psychologically, it’s just tempting to.. wait, wtf?.. did I just call my callback function ‘callback’. Hold on, let me fix that. Oh crap, I am duplicating this chunk of code all over the place, those bits better go in a separate module. And off I go down the rabbit hole…

This is a common pattern. It’s a lot like snacking — before you know it, you’ve eaten the whole bag of chips with nothing much to show for it except effectively managing your urge to snack. You’ve certainly scratched your itch to organize and streamline things, but the program still does what it did before. At the end of the day, a customer is unlikely to see an appreciable difference in features or performance as a result of this work.

They say work always fills the time allotted. Work on technical debt will fill the time just as well, but unchecked it will fill it with things a product manager would never want. Attempts to pay down technical debt without a clearly defined goal, say to refactor a particular piece of code, are usually a recipe for disaster. When it comes to technical debt, a blank check to roam wild will always be cashed for a sum you, as a product manager, would not be prepared to pay.

Technical debt is like a mound of sand with a few grains of gold

Chances are you will never have the time to get to every single grain, but there is definitely a chance that somewhere along the way you’ll stumble into something that rewards you for the effort.

From product management perspective it’s easy to dismiss technical debt as something that can be safely written down at some point in the distant future when you decide to go in a different direction and scrap the whole product or, and let’s be pragmatic here, when customers don’t line up for it down the block like it’s the next iPhone.

It may make engineering happy, but happy engineers do not a happy product manager make (it may be necessary but it’s not sufficient). After all, you are the voice of the customer, and the customer will never ask for consistent naming or decoupled, well-define modules, or a new build system. Why should you care is the question? What’s in it for you when you could be building out the next feature instead?

Every once in a while a miracle happens. Someone stumbles across a radically better way of doing things that brings very tangible customer value. It has happened to me a couple of times (e.g. when refactoring a module lead to an improved algorithm that dramatically changed the scalability of the entire system), and it changed my whole perspective on the value of paying down technical debt.

I went from seeing technical debt as something that was eliminated solely to satisfy the engineering’s drive to perfection to something that is like a high risk/high reward investment in your product. You invest time and effort in exchange for a rare but handsome windfall.

Paying down technical debt is a strategic activity

You are unlikely to get an immediate return for tackling technical debt, but it’s not a lottery stacked entirely against you. In fact, paying down technical debt can remove cruft that papers over serious software flaws, and these initial urges to fix something mundane often cascade into improving something that is very fundamental and important. Improvements in the code often drive improvements in the product.

Intentional technical debt is all about developers making a conscious decision to optimize for the present instead of for the future. Paying down the technical debt is the opposite — it’s about taking the time to optimize for the future. And when you frame it this way, it somehow ceases to be controversial. Compare that with product managers who put on their strategic hats to do market analysis and 3-year product roadmap or the same product managers rushing to answer customer emails. You couldn’t imagine one activity without the other.

In other words, if you do not make room for strategic work (either on engineering or product management side), your product is probably doomed. But how much room to make? Although I ‘d normally shy away from giving absolutely numbers, 20% strategic/80% tactical split on software development has worked for me. If a product is more mature, I’d say that the ratio of strategic effort (including the effort to pay down technical debt) can go up. If a product is still in pre-launch phase, I’d say 5-10% would be reasonable too.

Your mileage may vary, but the definite do’s are:

  1. Draw a box around the work at goes into paying down technical debt. Define exactly what gets addressed as part of the effort and go no further.
  2. Accept and make peace with the fact that you will never be done, but invest in paying down technical debt for those rare fortuitous discoveries that bring customer value.
  3. Find an avenue for engineering to escape the tactical grind to pay down the debt. This is part of your long term product strategy, and it’s not only good for the product, it’s good for the team.

4 responses to “Technical debt is a rabbit hole that goes deep, really deep…

  1. As a developer, I broadly agree with what you’ve written…
    …which is why I would tend to frame technical debt in terms of “gearing”; in the same manner as a company “gearing” it’s growth through raising finance.

    I recently read a piece along the lines of “technical debt? No such thing. The analogy is flawed because you can never be in positive equity, so it’s not valid and should be ignored”. I see this as BS – just another way of burying your head in the sand.

    Gearing is debt, yet most companies aren’t aiming to ever be out of debt because the additional finance funds growth, that outpaces the interest, unless the hearing gets to high. Ignore that, and you go out of business!

    Unsuitable or unsustainable gearing is the thing we’re trying to address with technical debt.

    We’re never aiming for zero technical debt as if we do, the cost of getting there is too high and prevents any real growth.
    Letting the debt get out of hand is even more costly as developing against and maintaining an indebted code base leads to a collapse of our ability to deliver high qualtiy in a predictable timeframe, the functionality is not able to move – it’s no longer liquid – your code ends up becoming insolvent!

    Many of the strategies for reducing tech debt are actually more akin to efficiency measures/cost savings that prevent further debt building up, than they are too the straight “paying down” of debt. This includes setting the code standards etc. against which code is judged in order to generate higher quality code.

    The “pay downs” come from refactoring/improving legacy code and as you rightly wrote, should be packaged up into “payment plans”, either as defined addons to a given sprint, or if you’re in a bad enough state, or just want to keep normal projects ring-fenced and uncontaminated by legacy debt – define sperate sprints and even independent projects as lump-sum repayments.

    The analogy still holds well even at that level, which is why I think it’s so useful, even if it’s so domes hard to get buy-in from the business on its importance.

    • Thanks, Will, for your comment. I am not very familiar with the term “gearing”, but I think I understand what you mean from the context.

      • A company’s Gearing is the ratio of its debt to its capital. (AKA Gearing Ratio)

        There are a few variants, but they all aim to measure financial stability (i.e. predictability) and liquidity (availability of cash assets – ability to react to change/flexibility).

        Basically, it measures the same sort of things we’re interested in when assessing and delivering software products.

        In the analogy “cash”/liquidity is pretty much the “agile-ness” (or agility!) of the code base(s) as assets.

    • “even if it’s so domes hard”
      =”even if it’s somtimes hard”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s