Estimation seems to have gotten a bit of a bad reputation lately.
One misconception I sometimes see is that estimation beyond just a few weeks is “not agile”. Another trend is that some people advocate against doing estimation at all mostly because they view it as a beginner tool, so by not estimating we are no longer beginners.
To me doing estimation or not does not really say much about “how agile you are”. The way I look at it is that we should estimate when the reasons to do so outweigh the reasons not to do so. That simple.
In some scenarios this also includes doing estimation of large backlogs.
So in this article I want to share what I see as some of the reasons FOR and AGAINST doing estimation of a larger backlog. You can then decide for yourself if your situation justifies doing it or not.
By large here I mean at least multiple quarters of calendar time and potentially multiple team years of work per team before what you are working on will become “minimally viable”.
In an upcoming second article I will share one method I have used with good results for how to do this in a reasonably short time, should you need this.
I hope this will be useful for you. So let’s get to it!
Why NOT
There are a lot of good reasons for NOT estimating a large backlog. We often hear a lot about these from agile champions. I guess I am one of those too! 🙂
- We would pay now for value much later. Spending time to estimate something now that we will benefit from much later can be a questionable investment. Especially if estimation is expensive we may feel we need our time and attention elsewhere right now.
- We need to understand to estimate, then re-learn much later to do the work. We often don’t remember what old and large items in a backlog are when it is time to start working on them. Spending time now to learn about them just to estimate forces re-learning later. This can be seen as rework and waste.
- We don’t have large backlog items. Some teams estimate mostly to make sure that items are small enough to promote good flow. When your team and stakeholders are used to breaking down large work into smaller pieces you may start doing this naturally without having to estimate at the team level. If this is the dominant reason for you to estimate in the first place you may at some point find yourself doing less of it.
- Large queues of work are expensive. A large backlog is basically just a queue of idle ideas that has a non-zero cost to just keep around. It is just sitting there and not adding value. You may argue that all it does is increase lead times and cost of making prioritization.
- Real needs change over time. The longer an item needs to wait the less likely it is that it will meet current needs. If your target is moving old items in your backlog tend to go stale. The more your target is moving the more true this will be. When the time finally arrives to pick up an old idea we probably need to reinvent it to meet current needs anyway. In this case the value of keeping an old idea around is just low. We should act on it now or discard it. If it was a good idea it will surface again!
- Focus on time to market and value of the content of the work. The cost of using an agile team is mostly fixed per time period. Most of it is salary. If our capacity is already fully funded and staffed we should focus more on maximizing the value of the work. We do this by making the work small and selecting the most valuable work so that we can get it out faster. We may not need to estimate much to do that consistently.
- If we allow a large backlog it tends to grow even LARGER. There are forces in organizations that tend to negotiate towards a higher demand than there is capacity. We need all the help we can get to keep our target small and by saying no to estimation of large work items we feel we effectively push back against these forces.
All things equal, it is often a lot better to actively keep your backlog SMALL and the number of items in it LOW. This promotes focused work and good flow. My experience is that it is always worth some discomfort to stretch towards this end, as it is hard but valueble to build the discipline to pull this off.
NOW – I have learned that all things are NOT always equal. In reality, there are a lot of tradeoffs beyond the above.
So let’s flip the coin and look at the other side.
Why YES
Yes. There are also a lot of good reasons FOR estimating a large backlog even in a “mature” agile organization. I have the impression that these are not talked about as much. I guess that is why I am writing this article. So let’s bring some of these forces and needs out in the open in a few variations so that they can be honored.
- When minimum viable is LARGE. Sometimes what you are building is replacing something old and large that is still supporting a profitable and growing business. If this is business critical software built decades ago it is not always possible to replace it piece by piece. The simple fact may be that this old thing was built as one massive monolith. Replacing it is more or less a lung and heart transplant operation and that is scary business. This is often the main reason it has not been replaced before. However, at some point the time comes to just to bite the bullet! In other cases you may be innovating in a very mature business and for your product to be viable compared to competitors it just needs to start off with a LOT of features. In both these examples you can still grow your product with incremental agile development and continously deploy to a staging environment. But you may not be able to convince real users to use it until you have built a lot of software. When will that be?
- There is abundance of good opportunities and a need to select and make bets. Typically, strategic opportunities outweigh the capacity to act or execute on those opportunities. Hard choices need to be made. Sometimes an earlier estimate is needed to say no to a good opportunity – to maximize the amount of work not done!
- Need to clarify what funding and staffing we need. It matters if we have a one or eight year backlog in front of us to become business viable at a competitive level. Not all efforts start out with all funding or staffing just “taken care of”. The larger the viable backlog scope the less likely this is to be the case. Both securing more funding and recruiting more people have lead times.
- Need to communicate with people who rely on our work and to manage expectations. Plans need to be based on real outcomes and be aligned with actual trends. If not, expectations can easily be mismanaged. When this happens trust fundamental for doing good work can break down. The larger the work scale the higher the impact of such trust failure. To base expectations on realistic forecasts a realistic estimation is typically needed.
- Need to embrace change while still doing long term planning. On the organization level our relationship with our surroundings often rely on an understanding of also what is possible at larger time scales. Longer term planning is often required to enable basic synchronization within an organization as well as with partners.
- Need to balance work to capacity at all scales. ROI and opportunity of hitting market windows can sometimes not be assessed in a meaningful way without at least rough estimates also at larger scales. If we don’t know size of demand we can’t level it to capacity. Estimating and breaking down large work helps us fit work to capacity.
- Need to create a sense of urgency to de-scope and prioritize. Sometimes a wakeup call is needed. Wishful thinking can be reeled in by making solid relative estimates based on “yesterday’s weather”. This is true also for larger backlogs. Sometimes you need insights driven by data that says that “this is way too large to fit” to hit home before needed decisions are made.
- Need to make the degree of uncertainty transparent to stakeholders and teams. In a changing world we know things are uncertain. But we can use good estimates to be transparent about just HOW MUCH. It is better to give “bad news early” rather than late. And when we frame potential future outcomes as options with limited degrees of quantified uncertainty we promote good decision making. Estimation at larger scales helps here.
So how much estimation do you need? Do you have important questions that need estimation to be answered?
Is it enough for us to estimate on the team level for the upcoming few weeks just to avoid over-commitment and promote focused work with good flow? Or are there enough good reasons for us to spend some time regularly to also estimate at larger backlog scales?
This is for you to decide.
Then – if you need to – see my next post for one way you can do large backlog estimation.
If you have your own favorite reasons for or against estimation, please share them in the comments!
