Agile teams often get their planning and priorities wrong

The doctor is in

Do you recognize any of these symptoms?

• I struggle to manage conflicting demands from multiple stakeholders.

• There is always someone disappointed that their business priority is not getting the attention they thought it deserved. This erodes confidence in the team.

• People are often saying things like "but this is #2 on the priority list; how can it have seen no progress this month?"

• We are constantly switching the team's priorities on the backlog and we can't finish anything because we are starting and stopping work all the time.

• We never have any time to pay back tech debt or get solid foundations in place.

If so, you are not alone. The problem is rooted in priority-based planning, and we have a solution that might help you.

Priority and planning

Backlogs

Most Agile planning methodologies are underpinned by the notion of the "prioritized backlog". For example, SCRUM speaks about two types of backlog: the Product Backlog and the Sprint Backlog.

"Backlog" is generally understood to mean that the team maintains a list of the current "known work" (refined to various degrees of granularity), in some kind of priority order.

It doesn't necessarily imply a particular means of describing that work.

In the wild, backlogs will often contain items that range from high level business requirements, to features, user stories, and individual tasks of differing degrees of size and complexity.

The stakeholders for those items may vary from end users, to business owners, to the team itself.

The "planning" process typically consists of prioritising that backlog from "highest" to "lowest", then dropping some or all of those items into "planning blocks", based on their position in that list.

The purpose and size of a "planning block", and the type of items that are dropped into it, depends on the type of plan. These might be upcoming sprints of some length (a week/month). Or you might develop a roadmap-style view of your higher-level targets for this month, quarter, half, year etc.

When working to deliver value, the team selects items from a backlog for the current iteration of work, drives them to some sense of "done" and then moves on to the next item or items.

You don't just see a backlog in iteration-based processes like SCRUM and those built on frameworks like Rational Unified Process; Kanban is, in essence, an example of this family that works directly from the sorted backlog, without intermediate groupings into blocks.

Competition and priority

In an idealized environment with easily prioritized work and single teams that are working on a single unit of deliverable value with a single backlog, this is great.

But the reality is that most teams have competing priorities across multiple discrete units of value for multiple stakeholders.

There is often little cohesion between the work required for each discrete piece of value. One unit of value is commonly delivered independently of another. However, it is all smooshed into a single "sprint" backlog for the team; and often a single "product" backlog for the entire solution.

Although we nominally prioritize this "overall backlog" a problem develops because competing priorities remain unclear.

When the business says "this is our highest priority" they rarely mean "to the exclusion of all else" - they want progress on many fronts, but would like to see "the highest priority" completed first (or "most").

What does this mean? And how do you decide what you should do in this situation?

Priority and Budget

When I have a decision to make, I always return to two questions:

1. What information do I have to make that decision?
2. What levers can I pull to implement that decision?

Let's treat each of these in turn.

Information

When it comes to shaping a delivery plan, the information we have is:

1. The business priorities, and the impact the planned work has on delivering value with respect to those priorities
2. The breakdown (to some levels of granularity) of the planned work and the estimates (to some levels of accuracy) of the cost and risks of delivering each of those pieces of work.
3. The size and capabilities of the team

The information we have is dynamic, and inter-dependent. What do I mean by that?

Some of the work itself is breaking down the remaining work to a level of granularity where we understand:

• what to deliver
• how to deliver it (and by whom)
• how much time it will take given those first two choices
• the impact it will have on business value
• the risks to successful delivery within those estimates

We do this in order that the business can reasonably be expected to set a priority for the work itself based on a Cost/Value/Risk analysis. They should not have to commit to a "black box" project; Agile principles say that we should bite off a small piece of work to characterise the overall job, to the point where everyone is comfortable with the cost/risk/value equation.

So that information feeds back into this breakdown-and-prioritise process. We repeat as work is broken down to the point where it can be completed, and as new work is discovered.

Levers

Once we understand what to deliver, we then have to determine what levers we have to drive the delivery itself.

The challenge here is how to fill up our work pipeline when all we have is a high-level "priority" from the business.

There are a couple of important considerations here:

1. We still want our team to have autonomy in terms of what they choose to do at the lowest level of granularity
   • The order in which we carry out highly granular tasks should be determined by those with the most knowledge of how they interact and the complexity of their dependencies.
   • We don't want the business to micro-manage highly skilled individuals; it is a waste of resource. If they still want to do this, there is a trust issue we need to address; perhaps by executing this planning and expectation management process better.
2. We don't want to become burdened by administration
   • We don't want to disrupt the way in which the team works within the sprint
3. We want to ensure that we deliver a "fair share" of value according to the current (perhaps ever changing) business priorities
   • The "highest priority" work cannot arbitrarily take all of the resources at the expense of the "lowest priority" work.
   • we need to keep the team focused on the needs of their users and stakeholders

This last point is very important.

Development teams often think of the work as a "narrow" front - complete Feature X¹, then start on Feature Y. Businesses rarely do. The stakeholders who want Feature Y may have (reluctantly) agreed that they will take second place to the stakeholders who want Feature X - but they don't expect there to be no progress at all.

Failing to take the lower-priority feature stakeholders along with you on the journey feeds in to a sense that resource is not shared "fairly" - especially when a new "highest priorty" item comes in to trump Feature Y down the line.

So what levers do we have that can help us to better prioritise work with these considerations in mind?

1. Priority
   • how important is each feature with respect to our Cost/Value/Risk analysis
2. Budget
   • how much of people's time (or other scarce resource) are we prepared to spend on each feature in this planning period

Budgeting

The priority alone is not enough to balance resources fairly across teams - especially if there is more work to be done than will fit into your "unit of planning".

A better approach is to take each prioritized unit of work and assign that a time budget² within your unit of planning.

For example: Features X, Y, and Z have been prioritized in that order, and initial rough estimates of cost and risk have been made.

So, we determine that we are going to spend 600 person hours on "Feature X" this year, and 300 person hours on "Feature Y", and 300 hours on "Feature Z"

The higher priority item receives more budget, and the lower priority less.

You can take this top-level commitment into each level of your planning.

So, when working on our roadmap, we determine that we are going to spend 150 person hours on "Feature X" this month and 20 person hours each on "Feature Y" and "Feature Z".

But next quarter, "Feature X" will get 400h, "Feature Y" 150h (completing "Feature X" and getting a start on "Feature Y"), plus 50h on "Feature Z" - maybe spiking its riskiest aspects, and addressing assumptions.

In the rest of the year, we will spend the remaining 50h on Feature X, maybe on maintainance or nice-to-have items. We will also round out Feature Y, and maybe work on Feature Z; but that is necessarily fuzzier as no doubt other priorities will have worked their way onto the backlog by that stage or we will uncover some hidden issues that will change the cost/value/risk equation and deprioritise other work.

You can see that the higher priority item has received the most budget, and it has been front loaded. The lower priority items are scheduled accordingly, but we intend to make progress on all accepted features.

When it comes down to the sprint planning, you can figure out how much time you've got on those features, and figure out which work to prioritise week-by-week to get to the most value, the fastest; or reduce overall risk the fastest, or whatever balance you are seeking to strike at that time - iterating in the fashion we described above.

Then, as each individual team member is figuring out how to spend the hours in their day, they can cross check the hours they are committing against the budget, and quickly see whether they are on track across all the commitments at their relative priorities.

Stakeholder communication

So what about this issue:

"My feature is #2 on the priorty list. How can it have seen no progress this month?"

First - we are much less likely to hit this problem. If a feature is re-prioritized it doesn't necessarily mean that work stops outright - it may be reduced to a "ticking over" level, or deferred to a later part of the plan, but, and this is the most important thing - the impact is clearly visible.

Because we ran this budget down from a top-level policy/prioritisation decision, it all rolls back up to reporting at the top level.

All the stakeholders can see whether the value delivered for the budget applied meets their expectations.

Providing, of course we show them.

A great way to do this is with a parking-lot style view.

This provides a concise summary of the piece of work.

Typically, it includes a ubiquitous name for the piece of work³, and the name/contact information for the business and delivery owners.

This is followed by a summary of the time budget across your roadmap granularity (this month, this quarter, this year), and how it has changed since the last planning period.

The next line captures the total amount of work to be done, and the amount completed (with an indication of whether they are each going up or down since the last planning period). It also highlights any key risks (usually just links)

Below that is a task burndown which indicates the expected completion date (or where work will stop) given the current budget.

Finally, there is a time-based cost line which indicates how many hours have been consumed on the work week by week, along with a cumulative total. This often has a "budget" line so you can see the cumulative total approaching the budget, and a projection of the point at which that total will be reached.

Each unit of work on the backlog gets its own item on the parking lot once it has budget committed. You sort the overall lot by priority, so you have a quick overview of relative progress in the whole.

This becomes the at-a-glance view that shows the progress on a unit of work.⁴

Risks of a budgeting process

The biggest risk in a budget-based process is falling foul of the sunk cost fallacy.

When it is clear how much has been spent on a unit of work, it is always tempting to apply the "one more heave" rule - and with each "additional heave" we have sunk more cost, approve more budget (and more focus) to get it "over the line". Before we know it the project is in a death march.

Fortunately, the budget process itself has a built in mechanism to mitigate that tendency. You simply allocate less resource (but not no resource) to this project, so that it can continue to tick forward, but the other work which now has a better value/cost/risk ratio can be afforded a greater share of the budget. It will get there in the end (or not!) but it doesn't de-rail the ongoing delivery of business value.

What happened to story points and gummy bears?

You'll probably have noticed that I've been talking about budgets in terms of TIME - whereas a lot of "agile" teams have been trained to think in terms of some abstraction like Story Points when they are estimating work, and not to talk about time.

What is a story point? It encapsulates a single measure of risk, effort, complexity for an estimate, which translates into a time measure via "flow" (the number of story points the team can deliver in a unit of time).

That's fine - if that's the way you like to estimate carry on doing so.⁵

None of that takes away from the fact that you actually have a finite number of people and a finite amount time in any given planning period in which to make progress.

If you operate within a budget like this, you need to go back and justify why you need more than you estimated, closing that information loop, responsing to the new situation and adding a decision point ("agile"), rather than progressing in a stately fashion "until it is done" without responding to the new inforamation you have available because you blew the original budget ("not-agile").

The lack of a feedback-loop and active decision-making based on new information by the stakeholders is why 90%+ of all teams that call themselves "agile" are, in fact, anything but.

This "budget" technique helps add the discipline that kind of team is missing.

Summary

The inputs to the planning process include:

• a backlog containing descriptions of the various units of value to deliver.
• an estimate of the cost of delivery, risk to delivery, and business value realized by delivery.

The output of planning is a prioritized backlog of work, with an attached time budget to be spent on each feature in the planning period.

This can be performed at various levels from the strategic allocation of resources across the business, to features in a roadmap, to tasks in a sprint plan.

The delivery process itself is iterative - some of the planned work may be designed to refine an item in the backlog, uncover new risks, or address existing assumptions. The output of delivery will feed back into the next bugeting cycle.

A parking lot view is a great way of providing a dashboard for the units of work as they progress through planning periods.