Technical Debt Management 101

First of all, make sure you read my previous technical debt management posts (how technical debt can be both good and bad) – this post is designed to address the issues described therein.

What can be done to prevent negative technical debt and ensure you are in control of positive technical debt? How do you prevent it becoming a problem?

The simple answer is that you can’t. All systems will have some planned and unplanned technical debt. Software development is a complex, fast moving business and developers are only human. Decisions and technology choices made in the best of faith will turn out to be wrong.

However there are some methods that can be used to minimize the risk and impact of technical debt. Here are some necessary elements of technical debt management:

  • Good system architecture
  • Abstract systems, so that switching out technologies or areas of the system can be done with minimal impact
  • Robust system management
  • Enough testing (automated or manual) and documentation to enable low risk re-engineering if needed
  • Shared responsibilities so that the system is not too dependent on individuals
  • Technical debt management
  • Awareness of areas of technical debt within the system and of the longer term impact of the compromises or decisions you are making now

Paying off debt

Dr._Strangelove_-_The_War_RoomSo, having followed all the good practices specified and having understood the nature of my technical debt, how should I pay it off when time comes?

Well, there are three ways and I will do my best to stretch the financial metaphor one last time to illustrate them…

Ongoing small payments

Make a commitment that with every feature introduced an area of technical debt will also be addressed. Make technical debt everyone’s ongoing problem.

Lump Sum Payment

Take everyone out of feature production for a set period and resolve the main areas of technical debt. Make technical debt everyone’s problem for a short time.

Dedicated payment plan

Create a dedicated technical debt management team who do nothing but resolve technical debt issues and run this alongside the development program. Create some people who have technical debt management as their only problem.

MVP vs Future Proofing

One of the buzz words at the moment is MVP (Minimum Viable Product) – the concept of only building the bare minimum that is needed to deliver the current requirement – like the XP concept of YAGNI (You Aren’t Gonna Need It).

Back in the mists of time when I first started developing, it was all about future proofing – building the framework into your software to allow for future changes to be introduced in a simpler way.

So which approach is right?

I often come across developers advocating elements of future proofing. The reasons advocating such are:

  • Future proofing builds a system based on solid foundations
  • Cutting costs by not doing this piece of work now will lead to increased costs later
  • The major impact of certain future events can be mitigated with a small amount of effort now

However, for me these advantages are outweighed by the advantages of the MVP approach:

  • The business will see benefit from the work much sooner
  • Future proofing is spending money now in case you need to spend it in future – the vast majority of future proofing efforts end up going unused
  • Future proofing can be made obsolete by non-technical issues
  • Changing business requirements
  • New technologies – how often have we spent days coding round an issue that then was included in the next release of the language?
  • Failure of third parties your system depends on

That said, I would still advocate good development practices as a way of minimizing the impact of change later on. Good practices like separation of concerns, loosely coupled objects, modular design, reasonable levels of abstraction, dependency injection etc. are all the building blocks that will enable you to create a system that is well placed for change.

Reduced functionality is no excuse for bad code.

Advertisements

Leave a comment

Filed under Opinions, Technical Debt

Negative Technical Debt part 2: Over-funding

This is a fairly common sort of technical debt, and again one that can be laid at the door of developers.

In financial terms this can be thought of as borrowing too much money and then paying high interest payments. In technical terms this is seen in the over-engineering of systems to build a level of complexity that is just not needed for the system being built.

This results in longer development time, larger amounts of bugs and much more complex ongoing maintenance. The analogy for this is building a hatchback and putting a Formula 1 engine in it (just in case one day it needs to enter a Grand Prix), and then giving it a lifetime warranty.

It was this sort of technical debt that the agile concept of YAGNI (You Aren’t Gonna Need It) was introduced to address. If it is not something that you need right now, then don’t build it. As we discussed in Positive Technical Debt part 2: Structured Finance, YAGNI can also lead to the development of technical debt, but that technical debt is positive and can be controlled.

This type of technical debt is the most negative form as it is very difficult to control. By its very nature it is something that often only becomes apparent when it is already is existence (if it was spotted in advance, it just wouldn’t be done).

Following agile practices, focused development, regular architectural reviews and experienced developers are all ways of dealing with this problem.

In the final post of this series I’ll talk about how to manage Technical Debt (subscribe). In the meantime, why not take a look at the whole Technical Debt series, where we discuss many forms of positive and negative technical debt?

Leave a comment

Filed under Opinions, Technical Debt

Negative Technical Debt part 1: Credit card debt

This is similar to the Structured Finance analogy except that it is introduced in an unplanned state. If the Structured Finance can be viewed as a business funding its startup by negotiating finance with the bank, this is closer to a startup funding itself entirely using a credit card.

When does this happen? Typically this is when systems are developed by inexperienced/incompetent developers without strong management/standards in place.

I’ve been involved in many systems that have areas that were

  • “developed by x and we don’t dare change it because we’ve no idea how it works”
  • have areas that were knocked up by the owner/his nephew who was doing A-Level computers and it would be too big a job to rewrite it now, or
  • were written in [insert language here] because someone wanted to have a play with it / was good at it and no one here now knows that language.

The first step to dealing with this sort of technical debt is to get a handle on the level of it. Get someone to sit down with the “credit card statements” and understand the interest that is being paid on this debt. Until that is completed there is no way to understand if it needs dealing with and what the cost of dealing with it is.

Once the position is understood, it can be dealt with in a similar way to the structured finance technical debt.

Other posts in this series:

Leave a comment

Filed under Opinions, Technical Debt

Positive Technical Debt part 3: Investor demanding payout

This is a slightly different problem related to Technical Debt and one that is more difficult to manage.

In financial terms I would liken this to having an initial investor who gives the business capital with no expectation of repayment. This capital drives expansion, but a certain point the investor demands full repayment.

From a technical perspective this is when a technology selection is made because it offers benefits in terms of business progression that allow you to get to where you need to be in the most beneficial way (e.g. because of speed of development, easy availability of staff, availability of other technologies/partners to integrate with). The system can be developed in a correct manner with no technical debt but at a point when technology becomes unviable and the only way forward is a fundamental re-architecture of the system.

The sort of events that could lead to this are

  • You reach the limits of the technology in terms of capacity for the level of usage that you now have. An example of this would be Twitter and Rails or Facebook and PHP. On a smaller scale I have worked with companies who wrote their first system using Access as a database and as the company grew hit the capacity of that platform and had to re-architect to use another database.
  • The platform/technology used is no longer available. This is a scenario that has become more likely now that people are using more 3rd party cloud based services. It is possible to wake up one morning and discover that a core part of your system is actually no longer available.

Again this is a positive form of technical debt, as it is taken on board with a view to getting benefit from a system as quickly as possible.

The way to deal with this sort of technical debt is largely down to good system architecture. Systems should be architected in such a way as to abstract away the connection with technologies as much as possible, making swapping them out as simple a job as possible. Systems should be built to horizontally scale to allow capacity problems to be dealt with by throwing tin at the problem. It can also largely be dealt with by technology selection; the majority of systems we write have a predictable growth plan (not many of us write Twitter!) and we can make a reasonable estimate of the longevity of technologies and technology providers.

Other posts in this series:

Leave a comment

Filed under Code, Technical Debt

Positive Technical Debt part 2: Structured Finance

This is the most commonly understood type of technical debt. This is when compromises are made in order to get a system up and running sooner.

The business benefit gained from this is obvious – if a system can be up and running two months sooner then not only does the business save two months of development costs, they also gain two months of benefit from having a working system.

“You Ain’t Gonna Need It”

This type of technical debt forms the basis of what the lean movement would call MVP (minimum viable product), which is the idea that the first release of any system should be the very minimum that will bring tangible benefit, and that this should be done in the quickest way possible. This is an extension of the agile philosophy of YAGNI (You Aren’t Gonna Need It) which says that you shouldn’t waste time building anything until it is actually needed. The MVP approach says that you should embrace technical debt as the best way to generate income and prove that potential benefit of a development.

The key thing to remember about technical debt is that, unlike financial debt, it never has to be repaid. Systems can exist, and continue exist for their entire lifespan with technical debt.

Paying back interest

There is, however, interest to be paid on this debt. Interest on technical debt is paid in two ways:

  1. The additional cost associated with introducing new features because of having to deal with the short cuts taken in earlier, and
  2. The reduction in benefit associated with the workarounds, manual processes and missing features in the system.

This interest will be increased if additional technical debt is being introduced with new features, and existing technical debt not being reduced at all. This is not an unusual situation in new systems where the demand for new features is intense.

So, when does this sort of technical debt become a problem?

With each new feature that is being introduced into a system there is a degree of benefit that will be gained, there will also be an additional cost associated with introducing the change because of technical debt issues within the system. At the point at which this cost exceeds the benefit gained, then you have a serious technical negative equity issue – at this point your technical debt needs to be addressed.

Remember that the additional cost associated with technical debt is made up of three elements:

  1. The physical extra cost because of the additional time and effort necessary to get the feature generated
  2. The opportunity cost associated with the additional time that it is necessary to wait until the benefit from the feature can be seen
  3. The competitive cost associated with being unable to be as quick to market with new features as your competitors

The key to managing this type of technical debt is to be aware of the technical debt that you are introducing, track it and be aware of the impact it is having.

More important is to be willing at a point before technical negative equity is hit to deal with it, and to reduce it in a controlled and proactive way. This can be done by maintaining a technical debt register or by creating technical debt items in your product backlog.

Other posts in this series:

Leave a comment

Filed under Opinions, Technical Debt

Positive Technical Debt part 1: A Director’s Loan

Essentially this illustrates one of the key points of technical debt – it never actually has to be paid back. Like a director’s loan it can happily sit there on the books with no expectation of repayment.

Image: Tobias Wolter

Image: Tobias Wolter

From a technical point of view this means that compromises are made to get a system up and running quickly or cheaply and those compromises just stay there.

This can only really be the case on smaller, single purpose systems that do not really get changed much. An example of this would be someone creating a quick timesheet management program to replace a complex manual process. Once created the system would be “good enough” and solve the majority of the problems that it needed to solve.

Make sure you check back soon or subscribe for part 2, and in the meantime you could always go back and read my introduction to Technical Debt, and why it’s not always bad.

Leave a comment

Filed under Opinions, Technical Debt

Technical Debt: It’s not all bad…

There is a lot of talk these days about technical debt and the evils associated with it.

While it is true that it can be a very negative force, it is also worth remembering that, when handled properly, it can be a positive force.

Like financial debt, technical debt can bring a lot of benefit to a business. Very few businesses can start trading without credit of some variety and keeping that credit to a minimum before income starts to be generated is the target of all businesses (unless you were a business in the dotcom bubble).

In technical terms there is a benefit associated with the building of all systems (whether this is actual physical income, increased efficiency or competitive advantage) and it often makes business sense to go into technical debt to get this benefit sooner.

However, there are also a lot of negative impacts associated with technical debt and these can come back to haunt you. The important part of this is to be aware of the technical debt you are taking on board, and do so by choice.

Here are some of the types of technical debt that may sneak up on you:

The departed genius
“x wrote this and no-one else understands it”

The next big thing
“x wrote this in a new tech that’s no longer supported, no-one else understands it”

The self sufficient developer
“x didn’t agree with / understand what y had done so he re-wrote the same functionality to run alongside it”

The uncommunicative developer
“x didn’t talk to y so didn’t realise they were working on nearly the same functionality so it is in there twice”

The grumpy developer
“x would never let anyone else see his code or work on his areas of the system”

The unhappy supplier
“That was developed by x before he fell out with the MD so we don’t have the source code and can’t make any changes”

In the next 5 posts I will go into some more detail about the types of positive and negative technical debt, so be sure to subscribe or check back soon!

Leave a comment

Filed under Opinions, Technical Debt