In Part 1 of this article, I talked about how Technical Debt negatively affects many aspects of Software Development. All right, so what are we still talking about — let’s roll up our sleeves and refactor the heck out of it, shouldn’t we!?
Refactoring is an ongoing process. It will never be done.
I think the first step is to realise that refactoring is not a task that has a clear beginning and a definite end. It’s not a one-time project! It is part of our daily work, and it should not even be the question if you should do it. Instead, think about how much time to spend on it besides the ongoing feature development, and where to start.
Not sure how to do that? Let me propose a simple 3-step plan than can help tackle technical debt.
Step 1: Reveal & Document
Do you know what the biggest pain-points in your software are? Usually, your developers have a good understanding of the problems in their projects, but how to make everyone (including the management) aware of it?
First of all, make sure that these issues get documented. I know this sounds very obvious, but if you don’t do it yet, define a workflow and make it happen! For example, ask your developers to write a ticket for each piece of technical debt they think they encountered. That could e.g. be the usage of the already mentioned anti-patterns or the needed update to a new version of a substantial library that was just released.
Technical debt not necessarily lies only in your code. Wrong architectural decisions can massively decrease the performance of your team. How to deal with those? They too have to get written down somewhere at first, so a ticket is a good start for the big ones as well. We will deal with those in the next step a bit more.
The above items go into the backlog (a list of tasks that your development team is planning to implement). Agree on a way to tag or organise technical debt tickets in the same way, so you can find them easily later.
Great, now you have the problems easily accessible for everyone. We’re getting there.
Step 2: Analyse & Prioritise
Now as you have a bunch of tickets – how do you decide what to refactor next? How do you know what value the refactoring brings to your project?
I’d suggest to start with setting up regular sessions (often called groomings), where the development team evaluates the issues identified as technical debt in step 1. If you already have groomings in place, the debt tickets will fit in perfectly.
During these sessions, your team should assess both the needed effort and the potential value of each item. While effort is — to some extent — relatively easy to predict, what about the value? This can end up in lengthy discussions.
Let’s assume there is this piece of business logic that is working just fine for years. It’s there, has no external dependencies (so updates are no problem), is battle-tested and after all the time pretty much bug-free. But still, our code quality analysers, i.e. tools which are scanning our source code relentlessly for violations to modern coding standards, complain that the code, well, sucks!
Sure, it’s super ugly and outdated from an academic point of view. But what value do we add by refactoring it to make it apply to rules which were introduced long after the code was written?
In my opinion, this should be one of the last pieces of code to be touched! Of course, there is value in making code easier to understand for developers. But the risk of introducing new bugs is higher than the benefit of having clean code which, in the end, only does the same thing. This risk is exceptionally high if your application is not covered with automated tests. And even if there are automated tests, these might not test every little part of the logic or are just not testing it properly.
Figuring out the real business value, i.e. an equivalent in money for paying back technical debt, is pretty tricky and would be surely worth an article of its own. Reducing technical debt is more about lowering effort than earning money, but with that extra level of uncertainty.
Good news: we don’t even have to go that far! Start simple and use T-Shirt sizes (e.g. from XXS to XXL) to have rough estimations for both business value and effort. While it is not easy to define the specific value of a refactoring task, it’s usually easier to put them into relation. Which one is bigger than the other? Which one brings more value? You then can identify which tasks are the most beneficial – you don’t have to put labels with concrete numbers on it at all.
To visualise it even better, arrange all evaluated tasks in a value-effort-matrix that you may already know from Lean Development:
A task which is is done quickly (low effort) but brings us a lot of benefits (high business value) is a Quick Win. Let’s do these first! Tasks of significant size (high effort) but with lower impact on our revenue (low business value) are Time Sinks, which should be avoided.
Keep the Maybes in your backlog, but with lower priority. Often enough, these little things might not gain much value, but are still somewhat annoying for the developer. Fix them when you got some time left, it’s often worth it.
Big bets often are structural problems hidden within your application architecture or even your whole infrastructure. Groomings are not enough here. Do workshops where your developers can put their heads together instead and come up with solutions. It’s nothing you can fix by doing some refactoring tickets now and then, it will require a new project and is thus out of scope for this 3-step-plan. That doesn’t mean you should not do them — it’s quite the contrary. You just can’t solve these items in your daily routine.
Step 3: Payback
Just tracking tickets in lists and drawing nifty graphs won’t help if there is no possibility to pay back the debt. So now it’s time to think about how much effort your team can, or better, should spend on it!
Finding the right balance between feature delivery and refactoring is crucial.
Refactoring alone will not convince your users to use your application and pay the bills in the end. The ratio between features and refactoring can and should change over time, depending on your current situation; remember those nasty deadlines!
Now comes the tricky part: The Business or the Product Management or whoever is setting the roadmap of features for your project surely won’t sacrifice development time so easily. Of course, the notion of refactoring being a sacrifice is wrong. You can use the arguments from Part 1 to convince them. If the business cares about to longevity of the application, they should understand.
You can also come up with a deal: the developers get some time to work on the most significant problems with priority (like a Refactoring sprint, for example), and afterwards the focus is on delivering some cool new features again.
Lastly, at some point you want to re-assess older refactoring tickets every now and then. Maybe your team got more experienced and would see things differently now, or the focus shifted, and you now realise that considerable changes in your architecture are more worth it than you initially thought.
That sounds all nice and not too hard to implement, but will it really work? Well, initiating the process might take a bit, because the developers have to accept it. It’s one thing to complain about technical debt (that is usually very easy), but working on a structured approach to solve it means extra effort for them.
Writing tickets is surely not the biggest fun for developers. You probably have to chase it a little bit. You have to remind them, e.g. by looking at the actual amount of technical debt tickets in your backlog together regularly. If the tickets have been tagged properly, it’s usually simple to get an overview, and you can check the progress. If there was none because no tickets have been added, remind your fellow developers kindly to write them.
Secondly, if the Business won’t give you enough time for bigger refactoring tasks, all you can do is document the debt, raise it again and again and wait for the moment when everything goes down the drain. In the end, it’s a Management decision. If good arguments won’t help, they have to learn it the hard way.
Let me be clear: I know how hard it can be to integrate all this into the daily work! It doesn’t mean you shouldn’t try it, though.
Conclusion & Outlook
Keep in mind that refactoring is an ongoing process and don’t stop at some point where you think you did enough. The simple 3-step process I explained above might help you, maybe even if you take just parts out of it so it fits to your company.
It surely is more overhead, but I think it’s worth it. You can’t solve a problem which you don’t fully understand or oversee. Our goal is to make the technical debt tangible.
Probably you don’t even need to go full ticket mode all the time. Let the developers agree on the boy scout rule: Leave the code in better shape than you found it! While working on new features, have a look to the left and the right and fix all these little things immediately. The actual task takes a bit longer, but over time those minor improvements will help a lot, and don’t require any management overhead.
In the end, every company has to earn money. It’s crucial to find the balance between maintenance and feature development which suits both business and technical needs.
In Part 3 (upcoming) I’ll get more technical, when I talk about the many manifestations of Technical Debt, and how to avoid accruing it. Stay tuned!