I talked about how moving the finish line is one of the most destructive things to do to a Scrum team, but I wanted to talk about another variation of the same problem, which is scope creep on backlog items in progress.
Responding to change
Agile is all about responding to change. Don’t get me wrong, I am not for one second suggesting that you should specify your requirements upfront before working on the backlog item.
Agile is also about iterative development. We take a thing, get a rough idea of what we want to build, build it and then iterate on it, making it better as we progress and figure out what we truly want.
The specific problem I am after today is about knowing when to allow change and at what resolution to iterate.
I think these are very important ideas that need to be understood to really be successful at Agile software development.
Change
To some people Agile means we allow change at any point in time. I think this view is a little too far on the extreme. There are certain points in an Agile process where change is welcomed and that depends on the process.
Scrum welcomes change up to the point the team has committed to the sprint. After that point changes to the scope of what will be worked on during the sprint are detrimental.
Kanban and Kanban like methodologies embrace change all the way up to the point that the backlog is in progress. At that point a significant change to the scope or focus of the backlog causes more harm than it provides business value. (In general)
What kind of changes are also important to define.
The kinds of changes greatly affect whether or not they can cause harm or be good. Changing the layout of a screen during the development of a backlog usually is not a big deal. Changing the way the screen functions usually is. Changes that add significant functionality also are harmful when the backlog is already in progress.
A little later on I’ll talk about some of the reasons for this. For now let's look at…
Iteration resolution
There is a prevailing misunderstanding of what iteration means and how it is valuable in Agile software development.
Many Agile practitioners think iteration applies only to the software package itself and to iterate means to add features.
This resolution is far too large. If the smallest iteration of your software is a complete feature, you probably have fatlogs, and you are going to run into trouble.
You may not ship features to a customer until they are completely evolved, but if you try to complete the evolution of features all in one sprint or cycle, you are probably going to bite off too much and miss out on the true value of iteration.
Just as your software should be iterating by adding features, the features themselves should be iterating by adding functionality.
Think of your software as a tree and features as branches. If you were to watch that tree grow, it would be pretty bizarre if whole complete branches just started popping up on the tree everywhere.
It is much more natural for the branches themselves to grow from little stubs into a large branch that might even have branches of its own.
Tying it back together
Given a backlog is already in progress and you have a good understanding of when change should be allowed in your process, and what resolution iteration should happen, it should start to become obvious why adding new requirements to that backlog is a bad idea.
If you take an existing backlog in progress and add new requirements to it, you're introducing change at the wrong point, and possibly trying to “poof” branches onto your software tree, instead of truly iterating.
It is much better to add a new backlog that will serve as an iteration of the existing feature than it is to try and tack on new requirements to the work already in progress.
But why?
Well, hopefully our discussion of change and iteration resolution has given you some idea of why, but there is much more depth to the question than that. Let's discuss some of them.
Psychology
There is a huge psychological portion of software development. We tend to be motivated by accomplishment and feelings of a “job well done.” We tend to be demotivated by goals changing out from underneath us and feeling like a job won’t ever really be done.
Let me give you a simple example. Imagine you tell your child that their chore for the day is to make their bed.
They go and do it, come back to you and tell you it is done, then ask for you to check it. When you check, you see that the bed is made, but you say “the floor also needs to have everything picked up.”
So they go and do that, it takes a little longer this time, because they have lost some motivation, but they finish and come back to you again and you see the floor is now cleaned up, but this time you say “the floor needs to be vacuumed.”
In this situation although each request is reasonable, the building of the requests on top of the original task has the effect of demotivating the individual and deprives them of their sense of accomplishment.
When you do this in software development, a similar thing happens. You also lose the trust of the team because the team starts to think that when the business says X they are really going to want X + Y + Z. The team starts to work much more slowly because it doesn’t really matter anyway. You do not want to operate in this mode! It is very bad!
Not letting evolution takes its place
If you try to cross a river by taking a huge leap across it, you are either going to get wet or you are going to make it. If the river is big, you will mostly get wet.
It is much better to cross a river leaping from stone to stone until you are all the way across.
Adding a requirement to a backlog in progress can be like making a big leap. By adding another backlog to bring you closer to the goal, you allow the current backlog to be a step, and the new backlog to be the next step.
It is a judgment call whether or not a change to an in progress backlog represents a new requirement which should be a new backlog or not. I would err on the side of making new backlogs rather than adding requirements though. I generally say if it will increase scope significantly it should be a new backlog. Either way the work is going to be done, so why not have your…
Estimating more accurate
Nothing kills historical estimations like scope creep. If you want accurate data, don’t add unaccounted for work. Squeezing new requirements into in-progress backlogs adds work to your historical data that is not properly accounted for.
It is much better to catalog the work and account for it than to tack it onto a place where you will not have a way to extract it back out when you are trying to analyze your data.
Consider also the loss in quality as a developer assigned to the expanding backlog starts rushing the work in order to get it done because they feel like they are going to be measured against the backlogs they completed, and work added to an existing backlog isn’t going to be counted in the metric.
Don’t forget prioritization here. If you tack a requirement onto a backlog in progress, you can’t prioritize that work. You may add that requirement at the expense of delaying other higher priority work in other backlogs. Make the new requirement into a backlog and you can prioritize it correctly.
Parting words
If you are a product manager, product owner, or business person reading this, give it a try. You will be amazed how much more happy and productive your team is if the next time you are working on a backlog with them and realize it needs to account for X and Y also, you announce that you will create two more backlogs instead of tacking it right on.
If you are a developer on an Agile team, you may not have even realized this problem might be sapping your motivation. You might not be in a position to do much about it, but you can always suggest that the new requirement be a new backlog.