Estimation is difficult.
Most people aren’t good at it–even in mundane situations.
For example, when my wife asks me how much longer it will take me to fix some issue I’m working on or to head home, I almost always invariably reply “five minutes.”
I almost always honestly believe it will only take five minutes, but it never does. Most of the time my five minutes ends up being half an hour or more.
But, in comparison to the world of software development efforts, my five minutes estimation is usually fairly accurate–it’s only off by a factor of six or so.
It’s not unheard of to have software development estimations be off by as much as one-hundred-fold. I’ve literally had an hour long estimation turn into two weeks.
But, why are software development estimations usually off by so much?
Here are the four biggest reasons I have found:
Reason 1: The unknown unknowns
This phrase was first uttered by former Secretary of Defense of the United States, Donald Rumsfeld. It basically refers to those things that we don’t even know that we don’t know.
By far, this is the biggest reason why software developers often flub at giving good estimations. It also happens to be the primary reason why I suck at telling my wife how long it will take me to get home–I don’t know about the distractions that I don’t know about yet.
Software development has a lot of unknowns. Some of the unknowns we know about.
When we first start a project, we might have a good idea that we need to store the data in a database, but we don’t know how we’ll do it. That is a known unknown. We know that we don’t know something that we’ll eventually need to know.
Estimating a known unknown is pretty difficult, but we can do a decent job of it if we can compare it to something similar we’ve already done before.
I don’t know how long it will take me to write this particular blog post, but I know about how long it has taken me to write other blog posts of a similar length.
What is really scary though, are the things that we don’t even know we don’t know yet.
These unknown unknowns sneak up on us, because we don’t even know they exist–by definition they are unknowable. It’s one thing to know that we have a gap in a bridge somewhere that we have to cross, it is a whole other thing to have to cross a bridge blindfolded and only find out about gaps when you fall through them.
Constantly, in software development, we are faced with situations where we have to encounter these unknown unknowns. There is no good way to estimate around them. The best we can do in these cases is give ourselves a lot of padding and a lot of rope so that we can climb out of any gaps in the bridge that we fall into.
Reason 2: Lengthy time periods
As if unknown unknowns weren’t bad enough, the deck is stacked against us even further.
Most software development estimations involve fairly long periods of time. This has changed somewhat with Agile development practices, but we still are often asked to estimate a week or more worth of work at a time. (Plus, let’s not forget those sneaky project managers who try to throw Agile projects into Microsoft Project anyway and say “yeah, I know this is Agile and all, but I’m just trying to get a rough idea of when we’ll be done with all of the features.”)
It’s fairly easy to estimate short periods of time–well, I guess unless it’s me telling my wife how long it will take for me to get off the computer. Most of us can pretty accurately predict how long it will take us to brush our teeth, write an email or eat dinner.
Long periods of time are much more difficult to estimate accurately. Estimating how long it will take you to clean out the garage, write a book, or even just to go grocery shopping is much more challenging.
The longer the period of time you are trying to estimate, the more that small miscalculations and the effects of known unknowns can cause an initial estimate to be grossly off target.
In my experience, I’ve found that estimating anything that will take more than about two hours is where things really start to go off of the rails.
As a mental exercise, try and estimate things of varying lengths.
How long will it take you to:
- do 10 pushups?
- make a cup of coffee?
- go to the convenience store and buy something?
- write a one page letter?
- read a 300 page novel?
- get the oil changed in your car?
Notice how the things that can be done in under half an hour are very easy to estimate with a high level of confidence, but as you go out further in time it gets much more difficult.
Most of the time when we do software development estimates, we don’t try and estimate short things, like how long it will take to write a single unit test, instead we tend to estimate longer things, like how long it will take to complete a feature.
Reason 3: Overconfidence
I’m pretty presumptuous when it comes to estimations. I usually think I’m very accurate at making estimations. My wife would disagree–at least when it comes to making estimations of time things will take me. History would probably tend to vindicate her viewpoint.
As software developers, we can often become pretty convinced of our ability to accurately predict how long something will take. Often, if the programming task we are about to embark upon is one we feel confident about, we can be pretty aggressive with our estimates–sometimes to the point of absurdity.
How long will it take you to get that feature done?
Oh, that? That’s easy. I can get that done in a few hours. I’ll have it done by tomorrow morning.
Are you sure? What about testing? What if something comes up?
Don’t worry, it’s easy. Shouldn’t be a problem at all. I just need to throw a few buttons on a page and hook up the backend code.
But, what happens when you actually sit down and try to implement the feature? Well, first you find out that it wasn’t quite as easy as you thought. You forgot to consider a few of the edge cases that have to be handled.
Pretty soon you find yourself taking the entire night to just get set up in order to actually start working on the problem. Hours turn into days, days into weeks and a month later, you’ve finally got some shippable code.
Now, this might be a bit of an exaggeration, but overconfidence can be a big problem in software development–especially when it comes to estimations.
Reason 4: Under-confidence
Under-confidence isn’t actually a word. I suppose that is because someone wasn’t confident enough to put it in the dictionary.
But, just as overconfidence can cause a software developer to under-estimate how long a programming task will take, under-confidence can cause that same software developer to overestimate a completely different task, which may even be much easier.
I don’t know about you, but I often find myself in situations where I am very unsure of how long something will take. I can turn a simple task that I don’t feel comfortable doing into a huge mountain that seems almost impassible.
We tend to view things that we’ve never done before as harder than they are and things that we have done before as easier than they are–it’s just human nature.
Although it may not seem like it, under-confidence can be just as deadly to estimations though. When we are under-confident, we are more likely to add large amounts of padding to our estimates. This padding might not seem all that bad, but work has a way of filling the time allotted for it. (This is known as Parkinson’s law.)
So, even though, when we are under-confident, it might appear that we are pretty accurate with our estimations, the truth is we may be wasting time by having work that might have been done in half the time fill the entire time that was allotted for it.
(By the way, if you are looking for a good book on Agile estimation, check out Mike Cohn’s book: Agile Estimating and Planning.)
Did I leave anything out? What do you think is the biggest reason why software development estimations are so difficult?
This post is the first in a series of posts about software development estimation. If you’d like to make sure you get the following post in this series talking about how to get better at estimation, make sure you subscribe here and you’ll get the post directly in your inbox.
Yes, that’s right. I am writing a blog post today about Scrum and Scrum Masters.
No, I haven’t lost my mind.
I just realized that out of everything I’ve written about Agile and Scrum, I never talked about what makes a good Scrum Master.
I’ve both been a Scrum Master and I’ve worked on a team with Scrum Masters and from both of those experiences I can tell you that there is much confusion about this particular role on a Scrum or Agile team.
Even the name Scrum Master has confusion around it, is it ScrumMaster or Scrum Master—you can tell, I prefer the latter.
So, let’s talk about why Scrum Masters exist in a Scrum team and what they actually should be doing.
Scrum Masters! What are they good for?
On some teams, unfortunately, it is absolutely nothing.
But, it doesn’t have to be that way. A Scrum Master actually serves a really important role on a properly functioning Scrum Team. (They bring the donuts or bagels in every morning, so the team can get actual work done.)
Ok, I am just kidding about that last part, but in reality it isn’t all that far from the truth. Let me explain.
A Scrum Master really is supposed to be the person who clears the path for the team so they can run as close to full speed as possible. The Scrum Master is sort of like the pit crew for a race car driver.
Without a Scrum Master, a Scrum team is slowed down by impediments which inevitably come up in any development project. It takes time and distracts the team to deal with these impediments, so the whole cadence of the team slows down unless someone external to the development process is moving the boulders out of the way.
So, really, the most important job of the Scrum Master is to remove impediments which may hamper a Scrum team from progressing on backlogs and getting their work done.
This isn’t the same thing as managing a project, because the Scrum Master isn’t deciding how and when things should be done. Instead, the Scrum Master is part of the team and the team as a whole is taking accountability for managing the project.
The Scrum Master also has the role of being the master of the Scrum process—hence the name. This is a tough spot to be in, but is a very important role that many teams neglect. The rules of Scrum are important to a successful Scrum team. One of the reasons why I started to write off Scrum as a process was simply because it was so difficult to get anyone to actually enforce the rules.
This is the job of the Scrum Master; he carries the big Scrum stick and he beats people over the head with it when they step out of line. He doesn’t do this because he is a big power-hungry bully. No, instead, he does this because he knows that the only way the team is going to produce their best work and not waste time arguing over process is if they all follow the process that was agreed upon from the start.
Scrum is intended to be more than just a way to develop software or organize teams, it is also a process that clearly defines what will happen, when it will happen and who will do what.
The Scrum Master is one of the most important roles on the team
It may seem, based on my previous description, that the role of a Scrum Master isn’t all that important to the overall performance of the team, but that is far from the truth.
In reality, the velocity of a team is more influenced by the Scrum Master than any other member of the team—with the exception of that lazy developer that breaks the build all the time and constantly falls asleep at meetings.
Even though the Scrum Master does not have direct control over the management of the team, the Scrum Master’s ability to both remove impediments and enforce the Scrum framework directly affects the team’s ability to get s!@# done.
A poor Scrum Master will let the team flounder and let outside influence distract the team from their work.
A poor Scrum Master will either be too timid or not care enough to force the team to obey the rules of Scrum, causing the whole platoon to go scampering off whatever direction they choose, rifles firing randomly in all directions.
I like to think of the Scrum Master as a guide who takes the team over rough terrain and shows them how to get water from tree leaves on their journey. Sure, the team could manage to bushwhack their way through the jungle without a guide, but it would take them a whole hell of a lot longer to do so—and they’d be much more likely to get eaten by a lion.
So, what should Scrum Masters actually do?
The answer is whatever needs to be done.
You know those gangster movies where some mob boss has a guy they call “the cleaner?” The guy that comes into a sticky situation and can hide a dead body, bribe the right cops, or just make someone disappear? If mobsters were following Scrum, that guy would be the Scrum Master.
The Scrum Master should be part of the team, but not part of the team. The Scrum Master should attend the standup meetings actively trying to spot impediments—especially the ones that aren’t mentioned by the team members, but exist beneath the surface of a problem.
The Scrum Master should ensure that all the Scrum meetings and processes flow smoothly. He should make sure that standups are being used for their correct purpose. He should encourage the team to hold each other accountable and he, himself, should hold the team accountable to what they promised to deliver.
The Scrum Master should be the guy (or gal), who makes things happen. He should know the right people to talk to and know how to get things done. The team should focus on the work, the Scrum Master should focus on the politics. If the team is dealing with politics the Scrum Master has failed.
Most of all, the Scrum Master should be willing to lay it all on the line—to take the hits for the team. Even though the Scrum Master doesn’t control the team and get to boss them around, they are his team and his alone. A good Scrum Master isn’t afraid to take full responsibility for the actions and performance of the team and step in the way of that bullet and take one in the chest if he has to.
Like this post? Sign up for my weekly email and I’ll make sure posts, like this one, are delivered right to your inbox once a week. Also, you’ll get lots of content that I only share with my email subscribers.
What is the hardest thing about writing code?
There are many common answers to this question:
- Learning a new technology
- Naming things
- Testing your code
- Fixing bugs
- Making software maintainable
The list goes on and on.
But as I reflect back on my programming career, and I’ve conversed with many new programmers that are learning the craft, I’ve found the single hardest thing about programming is learning the problem domain.
A familiar problem
In a good portion of my Pluralsight courses I show the viewer how to build the “Protein Tracker” application.
I am often asked why I keep demonstrating how to build the same simple application over and over again in each of my courses.
The answer is “familiarity.”
When I first started using the Protein Tracker example in my Android course, I was just looking for a simple example of an application that could be easily understood and implemented.
The idea behind the Protein Tracker application is that it allows a user to set a goal for the amount of protein to consume in a day. The user can add protein amounts which are added to a total protein count that is tracked for that user.
Very simple functionality, easily explainable, but most importantly, easily understood.
What I found with this simple application was that because it was so easy to understand, the focus was taken off of the problem domain and put instead on the technology.
Not only this, but as I reused this same exact application for teaching a variety of different technologies, it served as a reference problem domain that didn’t have to be re-learned, and provided a way to compare and contrast different technologies—I was getting this teaching mechanism for free if a viewer had already watched one of my other Pluralsight courses.
By creating a familiar problem domain, I found that both the tasks of me teaching a new technology and the viewer learning that technology were much easier, because it is very difficult to learn more than one thing at once.
So what am I trying to say here?
Simply that by taking away the problem domain, or making it so trivial that it is easily understood, I am able to make both teaching and learning easier.
Why problem domains are hard
Have you ever tried to put together a jigsaw puzzle that didn’t have any picture on it? How about one like this one, that has a very similar pattern repeated on it and is double-sided?
The reason why puzzles like this one are so hard, is because you can’t really see what you are trying to build very clearly. Normally when you put together a jigsaw puzzle you follow steps that might look something like this:
- Figure out what the major components of the picture are
- Sort the pieces by color or component
- Put together all the border pieces
- Put together each component of the picture from the piles you created
This all breaks down when you don’t have a picture with clear components that you can identify.
The same thing happens when writing code. Writing code is a lot like putting together a jigsaw puzzle. We put together code with the purpose of building components that we have taken out of the “bigger picture” of the problem domain.
The big issue is that many problem domains are like a puzzle with a blurry picture or no picture at all.
The real world is a messy place. Many of the problem domains we face as programmers are difficult to understand and look completely different depending on your viewpoint.
As programmers, we also are often not given complete information about the problem domain, so we don’t even have the information we need to understand it.
Just try and read the famous Domain Driven Design book and you’ll quickly see how complicated and difficult problem domains can be. (Great book by the way, although you may have to read it twice or three times—I certainly did.)
Programming is easy if you understand the problem domain
A long time ago, I worked for Hewlett Packard writing software for multi-function printers.
There was however something really interesting about the waterfall approach and the extreme amount of specification that was done before anything was built—it was very easy to write the code for a feature.
I remember writing a tab control for the user interface of a printer and having the complete pixel perfect specs handed to me before I began to write any code. I was also given all the possible use cases and told exactly how it should function and what it should do under just about every circumstance.
Guess how easy it was to write the code to produce this tab control? Super easy.
As much as I frown upon this approach for software development today, there is something interesting to think about here.
I was essentially given the entire problem domain in the form of a spec that was clear and unambiguous. I was easily able to learn that problem domain and because of it, I was able to write the code very easily as well.
Perhaps you have had a similar experience, not necessarily working on a waterfall project where you were given the spec, but perhaps on an Agile project where you took the time to clearly understand the problem domain before writing any code.
I’ve spent days trying to implement a feature only to finally go back and talk to a product owner and hash out completely how something should work and why it should work in a particular way, only to go back to my desk and crank out the code in a matter of hours.
The more and more I write code, the more I learn that understanding the problem is the most critical piece to the equation. It is very difficult to solve a problem before you know the question. It’s like buzzing in on Jeopardy before you hear the clue and shouting out random questions.
What can you do about it?
If understanding the problem domain is the hardest part of programming and you want to make programming easier, you can do one of two things:
- Make the problem domain easier
- Get better at understanding the problem domain
You can often make the problem domain easier by cutting out cases and narrowing your focus to a particular part of the problem.
What I mean by this is that it is often beneficial to take a part of the problem and fully understand that part before expanding the problem domain.
Games are really good at this. Look at most games today and you’ll find that you start with a very small problem domain. The first level is usually a tutorial that has a basic set of things you can do so that you don’t get overwhelmed. But, as you advance through the levels, you usually find they get harder and introduce new concepts that build gradually on what you know, until you understand a pretty large problem domain. (Starcraft is really good at this.)
The other choice is to become better at understanding problem domains. As developers, we tend to think that sitting down and talking to customers or business people who know about the problem domain is a waste of time. It is easy to fall into the trap of thinking you understand enough of the problem to get started coding it. Best to resist the temptation to “not waste anymore time talking” and make sure you understand a problem inside and out before you try and solve it with code. It is much more expensive and time consuming to do things over than it is to do them right the first time. I learn this lesson the hard way time and time again.
Quick update on my new product
I’m still not ready to unveil exactly what I am building, but I do have an active mailing list where you can sign up to find out when I release the product I’m working on to help developers get better career opportunities and market themselves.
So many developers don’t realize how much of an impact marketing themselves and branding can have on their opportunities. I’m hoping to help developers learn not only how valuable marketing and branding is, but how to do it most effectively.
If you are interested, please sign up. (I won’t spam you.)
Far too many Agile development projects fail. It is hard to even accurately measure the number of failures because so many software development projects end up “complete” and shipped, even though:
- They took far too long to build
- The quality of what we built was poor
- What was built was not what the customer wanted
- The cost to develop was more than it should have been
Over the years, I’ve worked on many different Agile teams and have consulted for and trained many teams on Agile development. In this time, I have found 5 common problems, that if not resolved, are very likely to cause a project to be a failure.
1. Not having a product owner
Of all the things that can cause an Agile software project to fail, not having a person that is ultimately the decision maker for the product being developed is the quickest way to ensure its demise.
It doesn’t matter if you are following Scrum, doing your own Kanban style project or something else; if you want your project to succeed, you need someone who can set its direction and make decisions about the product being developed.
Think about remodeling a kitchen. If you hired a bunch of contractors to come in and do the various jobs, like: plumbing; carpentry; flooring; etc, but you don’t have someone deciding what the actual finished kitchen should look like, you are going to get a random mess.
A few skilled contractors will probably be smart enough to find the right person to ask about what should be done, but it takes more than just making arbitrary decisions about where the cabinets should go to design a kitchen. You need someone to actually come up with a real design and vision and be able to modify that vision appropriately as the project progresses and problems inevitably occur.
It seems pretty nuts to spend a huge amount of money remodeling your kitchen, but not want to invest any time or effort in either designing the finished product or hiring someone to do it.
Yet, day in and day out, I see this exact behavior in software projects. I see companies investing huge amounts of cash in Agile development, but not appointing anyone to be the true owner of what is being built and to set the vision for it.
2. Not iterating
One of the key values that Agile development brings to the software development world is iteration.
What is iteration though?
You may think it means breaking your project into 2 week sprints or iterations. While doing this can facilitate iterative development, it doesn’t mean you are actually iterating.
Confused? Let me explain.
The key to iterating is to develop a product a little bit at time. It would be accurate to describe the process of iterating on a product as evolution of a product.
Whether you believe in macroevolution or not, microevolution, or adaptation is a proven concept. The idea behind evolution is that things change gradually over time. Those small changes add up to result in a big change.
Imagine how silly it would be if evolution worked the way most “Agile” software is built.
Imagine, if you will, a fish that swims in the ocean and has some little fish babies that are born with fully functional legs. Then, those legged fish babies grow up and have fish babies that have wings.
Probably the legs and wings wouldn’t do that fish much good, nor would they be designed properly, because instead of adapting and changing over time, they just suddenly appeared.
Features should not be built in a single sprint or iteration. It is as silly as legs showing up on a fish in a single generation.
Instead, features should be evolved and grow over time. A feature shouldn’t be pushed into a single sprint or iteration and then be done. A feature should start out small and develop and evolve over time as feedback is received and customers or the product owner tries it out.
Far too many times, I see Agile development projects mistake having iterations with actually iterating the development of the product.
Don’t try to ship a completed feature at once, let it evolve over time.
3. Not breaking things down small enough
The main reason why this is so important is because it prevents procrastination. Procrastination usually occurs when either we dread some large task that will be difficult or we don’t know what to do next.
If you can break a big project up into small parts, it will seem easier to accomplish and will have clear steps of progression.
I often see software projects where the person creating the backlogs or work items is not considering the work enough before giving it to the team.
I coined a term for these kinds of backlogs: fatlogs. Fatlogs are backlogs that are not broken down small enough and often are very vague in what needs to be accomplished.
Fatlogs are notoriously difficult to estimate and waste time when trying to explain and understand them. It is critical that fatlogs be broken down into smaller actionable backlogs before being given to an Agile development team or a large amount of time will be wasted.
Many times, I have found that the creator of the fatlog could have easily broken down the work it represents into several smaller backlogs that would be easier to explain and understand, even without knowing much about software development.
I often recommend to Agile development teams that they outright reject fatlogs and send them back up the chain.
“If you can’t take enough time to clearly state what you want, it must not be that important.”
This doesn’t excuse development teams either. Development teams should break down any backlogs they get into small tasks as well.
4. Not setting done criteria
What is the first thing a waiter asks you when you order a steak?
That’s right, they ask you how you would like it done.
If the chef doesn’t know what the done criteria is for your steak, the chef will have to decide what he or she thinks the done criteria is.
You may get back a well-done steak, or a rare steak, or something in between, depending on the personal preferences of the person cooking your meat.
This isn’t a good way to cook steaks and it isn’t a good way to cook software either.
In many software projects I often encounter lots of steaks being cooked but no one defining when they are done. The backlogs most often end up being “done” when time runs out.
It is very important to have explicit unambiguous done criteria for any backlogs being worked on by an Agile development team.
This means that the product owner should be defining some level of acceptance testing. It doesn’t matter if the tests are manual tests or fully automated tests (BATs), what matters is that some criteria is defined by which the team can evaluate whether they have achieved their goal or not.
Lacking this criteria is like a parent responding with “enough” when their child asks “how many more peas do I have to eat?”
Not having done criteria is demotivating and results in the finger pointing game of why the correct thing was not built.
I have found that if you tell someone exactly what is expected of them and what criteria they will be judged by, you get much better results than just throwing them in the ring, slapping them on the back, and saying “do a good job.”
5. Not letting the team be a team
Teams are strange organisms, especially Agile teams.
There are many dynamics that affect how teams act and interact. There are many ways to foster a healthy team and many ways to create unhealthy teams.
A healthy motivated team has synergy, a unhealthy gaunt team can be less effective than all of its members are on their own.
The key to having a healthy motivated team, is letting them be mostly autonomous. Regardless of your political persuasion, you would probably agree that when a country invades another country and sets up an unelected government to oversee the people, there are going to be problems.
The same thing happens in Agile development.
I’m not saying teams shouldn’t have accountability. But if you want to run a software project in an Agile way, you have to let teams self organize and self manage to some degree.
When big brother is always watching and interfering, team dynamics become very different. Naturally, teams usually develop their own cadence, leadership and roles (this is called norms.) However, when outside forces directly interfere with how the team does things, and they don’t have the power to decide their own fate, team members begin to realize that they need to look out for themselves.
Additional agile development resources
I’ve found it to be pretty difficult to find good resources on these topics, but here are a few books I’ve found useful that address some of the issues I described above.
- Succeeding with Agile: Software Development Using Scrum – This book is focused on Scrum but it applies to many different kinds of Agile projects as well. Mike Cohn and I usually agree on most Agile subjects.
- Agile Retrospectives - I’ve found this book to be useful for getting good idea for team retrospectives that can really help to break down some barriers and help teams learn to resolve issues themselves.
- Scrumban and Kanban and Scrum – Both of these books have excellent information about combining Scrum and Kanban and have great solutions for dealing with some of the problems I described above.
What do you think? Have I missed any of the major Agile project killers? Any useful tips for dealing with them?
Let me know in the comments below.
It seems just yesterday I was trying to push forward the idea of developing software in an Agile way, but somehow now it seems like that battle is over.
As if we won without a fight.
When I look around now, I don’t see software development shops doing big upfront design. I don’t see consultants knocking down doors to certify you as a Scrum Master.
It seems that we have now entered a phase where Agile is accepted as the default and now instead of everyone trying to pitch the idea of Agile, everyone is trying to fix their broken Agile implementations.
The funny thing is, still no one even knows what Agile is
The big problem with Agile from the beginning has always been trying to define it.
Pretty early on, this problem was solved by calling it Scrum. Scrum was something that was easily definable, and something you could get certified in.
Scrum was a set of rules you follow that makes you Agile.
At least that is how it was pitched too often.
I predicted that Scrum would die, and I am pretty ready to call that prediction as correct.
Sure, there are plenty of development shops still using Scrum today, but it isn’t really growing and less and less organizations are following it strictly. (I can’t back this up, just my feel.)
I am a pretty firm believer in most of the value of Scrum being that it contains a firm set of rules that doesn’t require debate or judgment calls. If most organizations are just taking the idea of a 2 week sprint and having daily scrum meetings, they are not likely getting much of the value out of Scrum.
But, the problem is that Scrum itself was never Agile. Scrum was a defined set of process, that if you followed, would give you the framework you needed to actually be Agile.
To me Agile has always meant stopping the BS about software development.
To me Agile meant stop building this plan that you know is going to fail and rules lawyering your customers to death to get them to pay for something they didn’t want, because that is what they agreed to.
To me Agile meant to instead try to develop software as honestly as possible. To go in and find out exactly what the customer wanted at the moment, try to build that thing and as openly as possible and as quickly as possible get further feedback to refine and improve. To focus on doing a good job and know that if you are doing that everything else will fall into place. To me that is what Agile has always been.
So when I say where is Agile now, I am probably asking a different question than most people
I have to ask myself: are software development shops doing what I define as Agile? Has that idea permeated the software development community as a whole?
I don’t think so, but I don’t think it has died, nor will it ever.
But, I have seen some things that make me hopeful.
I’ve seen a large amount of talk about MVP, Minimum Viable Product.
I’ve seen many start-ups launching MVPs and being successful doing so. And I’ve seen awesome companies like Buffer using this idea to build a product that is exactly what I want, because their plan is completely based on the customer and it adapts to the customer.
Why am I saying all this?
Simple, I think that what the world thought was Agile was two things:
- Iterative development
For the most part the software development world has ditched Scrum, at least the only form of useful Scrum, which is strict by-the-book Scrum, and adopted Scrum meetings and iterative development. Honestly, I could do without the Scrum meetings, because although they are a good idea, no one actually does them correctly.
So, in essence, we won the wrong battle and we did so with major concessions. But, that is ok, because what the consultants packaged up, certified people in and sold as Agile, wasn’t really Agile at all.
Instead the real Agile movement has been gaining traction and it isn’t being sold by consultants, it is being championed by small start-up companies that are producing products that are 100 times more weighted on the side of results in the results to employees ratio than traditional software development shops and they are calling it MVP.
This is where the true spirit and ideas of Agile live and thrive and as more and more of these companies become successful and more and more researchers dissect their results, they are going to find that these small software boutiques were the ones who were actually practicing Agile, because they were cutting through all the BS of software development and focusing and developing and building exactly what the customer wanted—tools, process, contracts, plans be damned!
There is a huge difference between a principle and a best practice.
Best practices are subjective and depend largely on context, while principles are eternal and universal.
After writing The More I Know The Less I Know, I received a few emails talking about how there are absolute best practices that should always be followed in software development.
I had already intended to write about principles, but that confusion made it clear to me that there should be a distinction made between best practices and principles. We don’t want to throw the baby out with the bath water.
Looking at some examples of best practices
First let’s take a look at some software development best practices, then we’ll contrast them to principles to better get an idea of the difference.
One of the most common best practices today in software development is the idea of unit testing. I’ve written about how I have my doubts about blindly following this best practice in the past, but whether or not we should follow it, is not what I am concerned with today.
Unit testing is extremely contextual. What I mean by this, is that almost anyone would agree that there are a certain set of circumstances that makes unit testing have value.
If you work in an environment where the execution of unit tests takes a really long time, or you are developing your software in a waterfall approach where you have a big upfront design and detailed requirements, unit testing starts to lose value rapidly.
But rather than get trapped into the argument of when unit testing loses its value, it is better to address when it has the highest value—we are much more likely to agree there.
Unit testing has the highest value when we are working in agile environments where changes are being introduced into a software system rapidly and refactoring is taking place. It also greatly increases in utility when you are able to execute and write the tests quickly, because that feedback loop makes it much easier to write the tests in a step by step approach, especially when doing TDD.
There are plenty of other best practices that have fallen out of favor, like heavily commenting code and documenting requirements with UML diagrams, but context also greatly played a part in the value of these practices.
When most developers wrote very short variable and method names, comments were really important. Before Agile processes became prevalent, getting detailed requirements upfront was critical.
But, most best practices are good!
Yes, you are right, most best practices do apply pretty broadly and are generally helpful in a large number of different contexts.
For example, it is considered a best practice to use a source control system and it doesn’t seem like there are many situations where this wouldn’t be the case.
So doesn’t that make it a concrete rule or a principle?
No, it is still too specific to be generally applied in all cases and the act of putting your code in source control does nothing to improve the quality of your software or software product.
If you were to blindly follow any best practice and not apply that best practice in a way that brings out the underlying principle, you would be very unlikely to actually receive any benefit.
You see, most best practices are actually derived from universally applicable principles that never change. That is why most best practices are good.
The problem is applying the best practice itself in no way assures the benefit of its underlying principle.
To put it plainly, there is something greater at work that makes it a good idea to check your code into a source control system. It is entirely possible to follow the action, but completely miss the spirit of the action.
More and more today, I see software development teams that are:
- Writing unit tests
- Using continuous integration systems
- Using source control
- Having Scrum meetings
- Pair programming
- Using IoC containers
Yet they are getting little to no benefit from it. Just a bunch more pain and hoops to jump through. The reason is simple…
It’s not the best practice that is effective, it is the principle behind the best practice
Principles are everywhere. They apply in all aspects of our life. You cannot go through the day without being affected by the results of 100s of different principles that have a constant influence on your life, just like the law of gravity does.
Gravity is actually a great way to understand principles. As far as we know, it is a universal force that is always in effect. It is impossible to escape the law of gravity, wherever you go in the universe it affects you.
Principles are like laws of nature except bigger. Principles are more like the laws of reality. Even though you may not be able to describe them fully or understand how they work, they always work.
Take for instance, the law of the harvest. Most people are familiar with this particular principle. It basically goes like this.
You reap what you sow.
How universal is this truth? How can anyone avoid it? How many times have you found yourself subject to this inescapable law about how reality works?
Many software development best practices are actually based on this principle. Think about best practices that have you make efforts to improve the quality of software early on in the process.
TDD or test driven development, is such a best practice. The basis of TDD is to introduce quality into the software development process as early as possible, so that the finished product is better.
If you apply the practice of TDD without understanding this principle, you are just following the motions and you won’t actually gain the benefit of the practice.
If you can’t understand at some level that the point of doing TDD is to sow some good seeds in your software that you will harvest later on, you won’t be writing the right kind of tests.
There is nothing magical about writing tests before writing code, but there is something valuable in purposely investing in upfront quality with the end goal of getting a big yield on that investment in the right season.
By the way, that is why I like Bob Martin’s book Agile Principles, Patterns and Practices in C#; it discusses many principles of software development that are timeless. Books like this one and the book I have mentioned probably 10 times in this blog, How to Win Friends and Influence People, are full of principles.
Also, check it out, you just learned what Agile really is. With principles in mind, now read the Agile manifesto. It was never designed to be a detailed process and set of best practices for developing software, it was always meant to be a recognition of a set of principles that guide software development.
So, just remember the next time you are arguing with someone over a best practice, or consider applying one to a project you are working on, if you don’t understand the underlying principle, no amount of ceremony and procedure will have the smallest amount of benefit.
The goal of software development is to solve problems.
At its heart, software development is really about solving problems through automation.
Many times we have the tendency to make software development about creating solutions. It may seem that these are the same thing, but there is a subtle difference.
The difference is the focus
When we are trying to solve problems, we are obsessed with the question of why. It is only by understanding why that we can know what to build.
When we are trying to build a solution, we are obsessed with the question of what. We want to know what to build so that we can build it.
This is fairly natural for software developers since the what is something we can control. Any fairly skilled software developer can build any what you can describe to them. Just like any skilled carpenter can build any type of wooden furniture you desire.
The difference is that software development is about more than just building things. A carpenter doesn’t have to focus so much on the why, to build you a piece of furniture. But a truly skilled carpenter will ask you why. A skilled carpenter might make a house call to the project to custom build an entertainment center suited to your needs.
A truly skilled craftsman of any craft, needs to know why.
It is odd that we constantly seem to neglect the why when building software (at least I do), primarily because we think we don’t have time for the why, when in the end solving the why is the only thing that really matters.
Do you really want a carpenter to build you a perfectly crafted entertainment center with plenty of custom shelves for all your music CDs when you don’t own any music CDs?
Why focusing on what doesn’t work
Consider for a moment what would happen if your GPS system stopped showing your route and the directions you were going to take ahead of time, but instead abruptly told you to turn right when you were supposed to turn.
One of the early GPS systems I used for navigation did exactly this. It was very frustrating! I would constantly shout at it that telling me what to do exactly when I am supposed to do it wasn’t any help. I needed some forewarning so I could switch lanes and prepare to turn.
Also, have you ever asked your GPS the question “why are you taking me this way?”
Focusing only on the what results in split-second decisions and avoidable mistakes. When you are focusing on the what you are not thinking, just doing.
Contrast this with a good GPS navigation system.
What makes a good navigation system good?
I’ve found that good systems will alert me far in advance to what my next move is. This gives me time to switch lanes or to have a better understanding of the bigger picture of the trip.
Now this allegory won’t carry us very far in the software development world. You don’t really need to know why your GPS system is taking you a particular route in order to get there, but it does demonstrate how focusing only on the immediate what can lead you astray.
Perhaps a more direct analogy related to software development is outsourcing.
If you’ve ever worked on a software development project that has had a portion of its work outsourced, you may have felt the pain of what happens when oftentimes perfectly competent programmers focus completely on the what and might not even have the slightest clue about the why.
I’m not trying to knock outsourcing, and I’m not even specifically talking about outsourcing to different countries. (The same problems exist in outsourcing whatever the conditions are.)
Oftentimes outsourced projects are treated like a set of blueprints that need to be built. The poor developers working on the outsourced project tend to get thrown under the bus when they build something based off just the what and can’t anticipate the why.
Why focusing on why is better
Focusing on the why is focusing on providing holistic cures to software development problems rather than treating each symptom individually.
Just like you’d like to have a doctor set a broken leg and put it in a cast so that it can be wholly cured, rather than give you some pain medicine to manage the pain, give you crutches to help you walk, and send you to therapy where you can learn to live without the use of your leg, your customers would probably rather you built something that actually is focused on solving their problem, not something that makes their symptoms easier to live with.
If we start building software from the what that is given to us, versus the why, we are at a distinct disadvantage because we only have the ability to treat the symptoms of a problem that we don’t even understand.
Focusing on the why is vitally important because it helps us to better think about and design the what.
Reuse is also much more likely to be present on why focused solutions than what focused ones.
Which software component is more likely to be adopted for reuse: a component that addresses a specific set of account balancing steps or a component that addresses the general problem your customer has of doing accounting?
Why focused solutions are much more likely to be shared across many customers, while what focused solutions are often very specific to a single customer.
Let’s not forget one of the most important reasons why it is important to focus on the why.
Missing the why is costly!
You can build the what someone is asking for, and you can do it perfectly, but at the end of the day if what you built doesn’t solve the problem (the why,) it’s going to have to be redone or thrown out completely.
You may think you are saving time by starting with the what, and you may actually build your solution faster by not taking time for everyone on the project to understand the why, but chances are you’ll build the wrong thing, and when you do, you’ll lose any time benefit you may have accrued.
It is very hard for a team to focus on the what. Focusing on the what tends towards segregating the responsibility of the team. The team members end up only having ownership for their own parts of the solution, rather than solving the problem at whole. So, when a team that focuses on the what ends up with a problem, finger pointing a shirking of responsibility inevitably ensue.
By instead focusing the team on the why, every member of that team becomes responsible for solving the problem rather than solving their part of the problem.
The most compelling reason to focus on the why?
For many problems, just having a thorough understanding of the problem takes you 90% of the way to the answer. I have spent countless hours trying to design and architect a clever solution to a problem without having a really good grasp of the problem that I’m trying to solve, only to find that 20 minutes spent actually fully understanding the why of the problem resulted in the answer becoming completely apparent.
How to know if your focusing on what
I’ve got a pretty good idea of how to know when I am focusing on what instead of why. Not because of any special wisdom on my part. No, it is because I fall down in this area quite a bit myself.
I’ve learned the hard way, and I am still learning how to identify when my focus needs to shift from what to why.
Here are some simple questions that you may find helpful to evaluate your own situation. I’d be glad to hear any others that you can can think of or use routinely yourself.
- Do you’re backlogs, stories or general work items state a solution instead of a problem?
- Do you know how to actually use the thing you are building? (Whoops, I am so guilty of this one so many times.)
- Can you define in simple terms the problem you are trying to solve? (Einstein said “If you can’t explain it simply, you don’t understand it well enough.”)
- Can you clearly define why the what your are building is needed and what purpose it will solve?
- Can you walk through the manual steps of the process you are automating, understanding each step and why it is necessary?
Taking the time to focus on the why instead of jumping into the what can be difficult to get accustomed to, but it is worth the effort. You’ll find much more satisfaction in your work when you understand the purpose it is going to serve, the true problem you have solved. In addition your customers will thank you!
What slows down the development of software?
Think about this question for a bit. Why is it that as most software evolves it gets harder and harder to add features and improve its structure?
Why is it that tasks that would have at one point been simple are now difficult and complex?
Why is it that teams that should be doing better over time seem to get worse?
Don’t feel bad if you don’t have an immediate answer to those questions. Most software practitioners don’t. They are hard questions after all.
If we knew all the answers, we wouldn’t really have these problems to begin with.
Regardless though, you’ll find many managers, business owners, customers and even software developers themselves looking for the answers to these questions, but often looking in the wrong place.
Process is almost always the first to be blamed. It stands to reason that a degradation of process or problems with the software development process are slowing things down.
Often there is some merit to this proposition, but I’ve found that it is often not the root cause. If your team is not sitting idle and the work that is important is being prioritized, chances are your process is not slowing you down.
Now don’t get me wrong here. I am not saying that these are the only two important aspects to judge a software development process, but I am saying that if generally your team is working hard on important stuff most of the time, you can’t magically improve process to the point of increasing productivity to any considerable order of magnitude. (In most cases.)
Often questions are asked like:
- Should we pair program or not pair program?
- Should we be using Scrum instead of Kanban?
- Should we be changing the way we define a backlog?
- Should we use t-shirt sizes or story points or make all backlogs the same size?
- Do we need more developers or more business analysts?
- Do we need to organize the team differently?
Now these are all great questions that every software project should constantly evaluate and ask themselves, but I’ve found over and over again that there is often a bigger problem staring us in the face that often gets ignored.
Let’s do a little experiment.
Forget about process. Forget about Scrum and backlogs and story points and everything else for a moment.
You are a developer. You have a task to implement some feature in the code base. No one else is around, there is no process, you just need to get this work done.
It might help to think about a feature you recently implemented or one that you are working on now. The important thing with this experiment is that I want to take away all the other “stuff” that isn’t related directly to designing and implementing that feature in the code base.
You will likely come to one of these conclusions:
1. The feature is easy to implement, you can do it quickly and know where to go and what to modify.
Good! That means you don’t really have a problem.
2. It is unclear what to do. You aren’t sure exactly what you are supposed to implement and how it fits into the way the system will be used.
In this case, you may actually have somewhat of a process problem. Your work needs to be more clearly defined before you begin on it. It may be that you just need to ask more questions. It may be that half baked ideas are ending up in your pipeline and someone needs to do a bit more thinking and legwork, before asking a developer to work on them.
3. Its hard to change the code. You’ve got to really dig into multiple areas and ask many questions about how things are working or are intended to work before you can make any changes.
This is the most likely case. Actually usually a combination of 2 and 3. And they both share a common problem—the code and system do not have a design or have departed from that design.
I find time and time again with most software systems experiencing a slow down in feature development turnaround that the problem is the code itself and the system has lost touch with its original design.
You only find this problem in successful companies though, because…
Sometimes you need to run with your shoelaces untied
I’ve consulted for several startups that eventually failed. There was one thing in common with those startups and many other startups in general—they had a well maintained and cared for codebase.
I’ve seen the best designs and best code in failed startups.
This seems a bit contradictory, I know, but let me explain.
The problem is that often these startups with pristine and well maintained code don’t make it to market fast enough. They are basically making sure their shoes laces are nicely tied as they stroll down the block carefully judging each step before it is taken.
What happens is they have the best designed and most maintainable product, but it either doesn’t get out there fast enough and the competition comes in with some VB6 app that two caffeine fueled not-really-programmers-but-I-learned-a-bit-of-code developers wrote overnight or they don’t actually build what the customer wants, because they don’t iterate quick enough.
Now am I saying that you need to write crap code with no design and ship it or you will fail?
Am I saying that you can’t start a company with good software development practices and a clean well maintainable codebase and succeed?
No, but what I am saying is that a majority of companies that are successful are the ones that put the focus on customers and getting the product out there first and software second.
In other words if you look at 10 successful companies over 5 years old and look at their codebase, 9 of them might have some pretty crappy or non-existent architecture and a system that departed pretty far from the original design.
Didn’t you say something about pulling up your pants?
Ok, so where am I driving at with all this?
Time for an analogy.
So these companies that are winning and surviving past year 5, they are usually running. They are running fast, but in the process of running their shoelaces come untied.
They might not even notice the shoelaces are untied until the first few times they step on one and trip. Regardless they keep running. And to some degree, this is good, this is what makes them succeed when some of their failed competitors do take the time to tie their shoelaces, but those competitors end up getting far behind in the race.
The problem comes pretty close to after that 5 year mark, when they want to take things to the next level. All this time they have been running with those shoelaces untied and they have learned to do this kind of wobble run where they occasionally trip on a shoe lace, but they try to keep their legs far enough apart to not actually step on a shoelace.
It slows them down a bit, but they are still running. Still adding those features fast and furious.
After some time though, their pants start to fall down. They don’t really have time to stop running and pull up those pants, so as they are running those pants slip further down.
Now they are really running funny. At this point they are putting forth the effort of running, but the shoelaces and pants are just too much, they are moving quite slow. An old woman with ankle weights power walks past them, but they can’t stop now to tie the shoelaces and pull up those pants, because they have to make up for the time they lost earlier when the pants first fell down.
At this point they start looking for ways to fix the problem without slowing down and pulling up the pants. At this point they try running different ways. They try skipping. Someone gets the idea that they need more legs.
I think you get the idea.
What they really need to do at this point though is…
Stop running, tie your shoes and pull up your pants!
Hopefully you’ve figured out that this analogy is what happens to a mature system’s code base and overall architecture.
Over time when you are running so fast, your system ends up getting its shoelaces undone, which slows you down a little. Soon, your system’s pants start to fall down and then you really start to slow down.
It gets worse and worse until you are moving so slow you are actually moving backwards.
Unfortunately, I don’t have a magic answer. If you’ve gotten the artificial speed boost you can gain from neglecting overall system design and architecture, you have to pay the piper and redesign that system and refactor it back into an architecture.
This might be a complete rewrite, it might be a concerted effort to get things back on track. But, regardless it is going to require you to stop running. (Have you ever tried to tie your shoelaces while running?)
Don’t feel bad, you didn’t do anything wrong. You survived where others who were too careful failed. Just don’t ignore the fact that your pants are at your ankles and you are tripping over every step, do something about it!
Pay attention young programmers, this is the most important piece of programming advice you will ever hear.
Well perhaps not, but it might be the most important piece of programming advice you hear today.
“Switching gears is grinding gears.”
I’ve been doing this programming thing for quite a while now, and I have come to realize that the biggest thing that saps my productivity is switching gears.
What do I mean by switching gears?
Many different contexts
Switching gears can apply to a variety of different contexts, but it is basically whenever you are working on some task or technology and have to stop what you are doing and either do something else or change the type of thing you are doing.
This is really a bit different than multi-tasking. Multi-tasking is really trying to do more than one thing at once. It usually involves a large amount of rapid context switches, but for this post I am more concerned about the general idea of breaking rhythm.
I think it is easier to speak about this subject if we talk about some specific applications of the idea and why they are harmful.
In the Agile world of software development today, we often are required to switch from problem domain to problem domain as many of our iterations contain mixes of different features on different parts of the system and bugs to be fixed.
Have you ever been in the situation where you were just beginning to understand how some area of the code base worked, or how to work with a particular customer on a particular problem space and then had to switch to something else and start all over again?
Even when you switch back to the original context you were working with, if enough time goes by, it becomes new again. You have to relearn much of what you had learned the last time and just when you begin to hit your groove, the cycle begins again.
This form of switching gears is at the very least frustrating, and at the most a complete demotivator and productivity sapper.
Unfortunately for most developers, this problem is out of your immediate control. But, take note product owners and project managers, there is a reason why iterations should have a goal.
If you are in the precarious position of being a developer pinned against the ropes, try to make some noise and let your management and product owners know that your team will be much more efficient when you aren’t switching gears all the time.
In my experience, the results of replacing this constant context switching with the synergy of a common goal and related features in a segment of time has an astounding effect on productivity that is hard for any serious businessperson to ignore. So speak up!
Switching technology is painful!
It really isn’t worth trying to fight this because you are going to be going against the technology grain to do so. Although, I do suppose this may be one of the reasons for the recent popularity of Node.js.
I’m not saying to not learn new things. It is very important to always be learning.
What I am saying, is to try to find some kind of rhythm with the technology stack you are working with and try not to switch that up.
Technology specific training can really help here as well. I for one, need to learn JQuery better. The problem is that when I am working on a web based feature, I am not forced to learn JQuery because I am not in that context long enough.
So what do I do instead?
I waste time Googling for answers. I know that I have a short coming here and I need to just bite the bullet and spend some dedicated time to really thoroughly learn JQuery, because by Googling for little pieces of it at a time, I am not really making much headway and the context switch ends up stealing what I do learn from my memory.
One more aspect of this is the idea of focused teams. Many software development groups do not like to specialize their developers onto one focus area. I agree whole-heartily with the idea of non-specialization.
But! There is huge benefit to be gained by keeping the same group of developers working on a specific set of technology or part of the code base for a set period of time. I’ll talk about this a bit more, towards the end of this post, but the basic idea is that it takes time for people to find their groove.
I think it is optimal to break any large team up into smaller technology area focused teams that regularly rotate every 3 months or so. The idea here is that you give those teams enough time to get good at what they are doing and actually commit what they have learned to memory, but you rotate them often enough that they don’t end up becoming specialists which are unable to see the whole picture.
This one typically isn’t an issue, but it can be depending on the environment that you are working in.
Teams need enough time to go through that forming, storming, and norming set of phases. If you are constantly interrupting this process by switching around team members, you are never going to get the team to act as an entity of its own.
Teams are generally more efficient than individuals, because they benefit from synergy, when 1 + 1 = more than 2.
But just like a big truck taking some time to get up speed, a team takes time to get going. Also like a big truck, a team can gain momentum that individual developers seldom can achieve.
A smaller aspect of this is pair programming. I suppose that many have been successful switching up programming pairs once a day or so, but I suppose that more teams have been successful switching up pairs at longer intervals.
For me, this interval varies. Sometimes, I feel like I need to stay paired with a teammate for longer than 2 weeks, which is our regular interval, sometimes 2 weeks is fine. It depends on what you are working on and how much momentum you have.
They key here is to make sure that you are giving teams enough time to take up their own flag and stake their territory. Teams can take some time to find their common goal. Self-direction can really help with this.
Ever tried to turn a flat head screw with a screwdriver, but it doesn’t quite turn, because you can’t seem to get the head of the screwdriver into the groove of the screw?
You twist that screwdriver around a bit until finally it slips into the groove. Then when you turn the screwdriver the screw turns with it.
As humans, we tend to be like screwdrivers, we have to find our groove. We all have this adjustment period where we are fumbling through things.
It is very important to make sure that we aren’t switching gears so often that we are not actually turning any screws once we are in the groove.
Regardless of what kind of context switching you are doing—whether it is problem domain, technology or teams—it is important to make sure that you are not spending 90% of your time finding the groove and only 10% of your time “in the groove.”
Depending on what you are doing and what the context switch is, the proper amount of time before switching contexts is going to vary, but I think it is very important be aware of this phenomenon and plan around it. If you don’t you are going to end up spinning your wheels, which is unfulfilling to say the least.
One of the biggest challenges in breaking down backlogs is knowing how to split up the work from a backlog into right sized pieces.
I’ve already talked about the concept that smaller is better, but I we haven’t really addressed the decision of how to actually divide a backlog up to make it smaller.
The default path
Most developers trying to break down a backlog into smaller chunks will automatically head down the path of using a “horizontal slice.”
This is how we tend to think.
What do I mean by a horizontal slice?
A horizontal slice is basically a slice through the feature or backlog that horizontally divides up the architecture.
Most things are built this way.
If you were to build a house, you would probably start by slicing up the project horizontally.
You would first pour the foundation. Then put up the walls. Then put on the roof and many more steps, leaving the finishing work for last.
This same thinking usually gets applied to breaking up backlogs in Agile development.
It would seem pretty silly to build a house where you finished one room completely at a time.
Agile software development is different
There is a distinct difference though, between developing software in an Agile way and building a house.
The big difference is that in Agile software development, true Agile development, you don’t know exactly what you are going to build until you are done building it.
With a house this is rarely the case.
With a house, you have some blueprints that you have drawn up ahead of time. You know exactly where each wall will be and where each outlet will be. You may have even built houses before that are very similar.
When building software, unless you are taking a waterfall approach and planning everything upfront, you don’t know what you are really building until you are done.
Before you object to this statement, consider this:
This is the point of Agile development.
Agile means responding to change.
Building a house, you do not expect the customer to say:
“Hmm, yeah, I don’t really like that wall there.”
“Actually, I am thinking we are going to need 5 bedrooms now.”
In software development, you are expecting statements analogous to the above!
So what is vertical slicing?
Simply put, building one room at a time.
But it’s not functional! Who wants a house one room at a time?!?
Correct! It is not functional as a house, but we can pour more foundation, change how we are going to do the rest of the rooms and even knock down the walls and start over without incurring a huge cost.
The point in building our software “one room at a time,” is that we are giving the customer a chance to see the product as it is being built in a way that matters to them and enables them to test it out.
Sure they aren’t going to be able to live in it until it is all done. But, they will have the ability to step into a room and envision it with all their furniture in there.
Customers don’t care about foundations and framed in walls. As a developer, you might be able to look at some foundation and framed in walls and envision what the house will look like, but the customer can’t and worse yet, it can’t be tested.
Vertical slicing in software development is taking a backlog that might have some database component, some business logic and a user interface and breaking it down into small stepwise progressions where each step cuts through every slice.
The idea is that instead of breaking a backlog up into the following:
- Implement the database layer for A, B and C
- Implement the business logic layer for A, B and C
- Implement the user interface for A, B and C
The backlog is broken up into something like:
- Implement A from end to end
- Implement B from end to end
- Implement C from end to end
Sounds easy enough, why the debate?
Because it is NOT easy.
I’m not going to lie to you. It is MUCH easier to slice up a backlog horizontally.
As developers we tend to think about the horizontal slicing when we plan out the implementation of a backlog.
We tend to want to implement things by building one layer at a time.
Thinking about how to break apart a backlog into vertical slices requires us to step outside the understanding of the code and implementation and instead think about the backlog in small pieces of working functionality.
There is almost always some progression of functionality that can be found for a large backlog.
What I mean by this is that there are almost always smaller steps or evolutions in functionality that can be created in order to produce and end result in software development.
Sometimes the steps that are required to break up a backlog vertically are going to result in a bit of waste.
Sometimes you are going to purposely create a basic user interface that you know you are going to redo parts of as you implement more vertical slices.
This is OK!
It is better to plan small amounts of rework than to build up an entire feature one horizontal slice at a time and have to rework huge parts of the feature that weren’t planned for.
So what is the benefit?
You might be thinking to yourself that this sounds like more work without much benefit. So why would I bother to break up a backlog vertically?
Is it really that important?
I’ve already hinted at some of the benefits of slicing things vertically.
The true impetus behind vertical slicing is the very cornerstone of Agile methodology. It is about delivering working functionality as soon as possible.
We aren’t going to cover the whole reasoning behind this idea in Agile development. I am assuming that you already subscribe to the idea that delivering working functionality as soon as possible is important and valuable.
Based on that premise alone, you can see that horizontal slicing is in direct violation to one of Agile methodology’s core tenants.
It is interesting to me how many people are huge proponents of breaking entire systems up into functional pieces that are delivered one piece at a time, but are so opposed to doing it at the micro scale when dealing with individual backlog items.
If you are opposed to what I am saying about vertical slicing, you really have to ask yourself whether or not you truly subscribe to the same idea applied at the larger level, because there really isn’t a difference.
As always, you can subscribe to this RSS feed to follow my posts on Making the Complex Simple. Feel free to check out ElegantCode.com where I post about the topic of writing elegant code. Also, you can follow me on twitter here