Working in software, you’ll find three general models for process workflow: SDLC/Waterfall, Kanban, and Scrum. If you’re on a Waterfall project, stop what you’re doing and go jump out a window. (Kidding. Mostly. Use a parachute….) If you have a good management team and a good set of colleagues, even if you’re on an SDLC/Waterfall project, the idea would be to move the operational strategy more in the direction of agile development processes. But which one is the right one? Why doesn’t Waterfall work in software?

If you’re on an agile project, you should note that there’s no such thing as a cookie-cutter development process. That’s what makes it agile; it’s flexible and malleable so that as the engineering team evolves to meet the needs of the customers, the process can evolve with them. One of Waterfall’s biggest weaknesses is its extreme rigidity, making it ill-suited for any agile project you’re doing.

In truth, Waterfall works for certain kinds of projects, but those projects don’t tend to move software around, and they tend to have a predictable process. Nothing about software is ever predictable, and this is generally why Waterfall does not work in software. Waterfall thrives on predictability.

Regardless, for any solid process to work, a crucial foundational point has to be in place. For instance, Kanban works to limit Work in Progress. Scrum works to limit Scope. Both work to tighten feedback loops between idea and production.

Waterfall: Not For Programmer Consumption

Don’t get me wrong, I’m sure some programmer somewhere got to participate in a Waterfall project, and everything was amazingly well done. Probably. Waterfall as a project management discipline works pretty well in fields where you have a good idea of the estimated time going into the work. If you’re building a house, Waterfall is probably OK, and even preferred, because you want to get the plumber on site at the right time—plumbers are expensive.

On a software project, the unpredictability of doing something brand new that was likely never done before makes it very difficult for Waterfall to work. Since this article is directed at more advanced programmers, I’m not going to spend time explaining how it works, so if you really want to know you can look here.

The reasons why Waterfall doesn’t work well for software include:

1. Rigid timelines that make adjusting for unknowns impossible.
2. An inflexible scope that doesn’t respond to changing customer needs.
3. A long feedback loop between idea and measurement.
4. Not being amenable to measurements/adjustments/reactions.

Agile: A Multifaceted Spectrum

On the other end of the engineering spectrum is the agile development concept. There’s two general mechanisms for this: Kanban and Scrum.

What's a Kanban?

According to Google’s dictionary, the literal definition of Kanban helps explain how the mechanism works and is stated as “a Japanese manufacturing system in which the supply of components is regulated through the use of a card displaying a sequence of specifications and instructions, sent along the production line.” Also, Google tells us the origin derives from “Billboard” or “Sign.”

Car Manufacturing?

The general idea of Kanban is that you have different columns on a board that represent issue statuses. And your whole team works to limit WIP (Work in Progress) in any one of those columns.

The Kanban process centers around bottleneck identification and elimination. Each lane on a Kanban board serves as a single step in the production process. If any one lane starts to fill up, the entire team stops and figures out what's causing the blockage. This was adopted directly from car manufacturing. If one station in the output becomes a bottleneck, everyone works together to eliminate that bottleneck to optimize throughput.

If you have ever used Trello for team or personal management, it’s Kanban in a nutshell. You put cards on a board, and follow them to completion. Even though Trello is pretty darned simple, I actually prefer an even simpler setup using OneNote.

Here’s my personal Kanban board on OneNote. It has a few more than those three statuses in it, and it combines home life and work life, but it’s still simple to use and understand:

That's Oversimplified

While Kanban depends on the specific nuances of the team you're on in order to work, in general it operates best in a fully-automated, continuous delivery environment. It's much more difficult to run Kanban on a team that depends on manual deployment because one of the columns would be deployment, and everything would get stuck there.

And a column with 300 cards in it is hardly easy to keep track of.

If you want more information about how Kanban can work effectively, there’s an almost non-fiction book called The Phoenix Project that does a really good job explaining how to transition from a Waterfall mindset to a continuous delivery mindset. It’s fun, and a bit silly—but it definitely teaches the lessons of the continuous delivery concept well.

We Can’t Really Kanban: Scrum to the Rescue?

If you absolutely cannot build continuous deployment into your process, you're going to have better luck attacking your project using Scrum.

If you want to spend time understanding how Scrum is supposed to work, these guys did a fine job explaining the concept.

The most important takeaway for making Scrum work is frozen scope. After you plan a sprint, nothing else gets in until the next sprint. That makes the sprint-planning meeting one of the most essential days in a Scrum sprint. Get the sprint scope right, and then don’t let anybody change it. Period.

What About Fires?

The best analogy I can come up with for Scrum/Kanban is putting out a large forest fire.

You have to have two (or more) teams. One focuses on the parts of the forest that are already burning (Kanban) until the fire is out. The other focuses on the long term plan for preventing an as yet un-burnt forest from ignition (Scrum). The latter takes a different kind of planning and is proactive instead of reactive.

The best blend I see working well is a Kanban team dedicated to putting out the spot fires of escalated support, and a Scrum team dedicated to moving new scope and bug fixes forward by integrating product management into the single backlog.

More than anything else, for agile development to work in the general sense, you’ll need to work with your peers and your product team to have a single, unified, and prioritized backlog. As long as the people managing your project get to decide what gets worked on first, the engineers get to decide how. A well governed backlog is critical for agile of any kind to work.

Should We Try Estimation?

Planning poker is a pretty critical part of how Scrum operates if you take the process literally. Getting the right amount of scope planned into a sprint takes a pretty delicate balance. Whether or not you include estimation as part of this process, the general idea is to get better at planning out each sprint so that you only put as much into it as you can get done during the sprint timeline.

And, if you do decide to use estimation, by all means focus on complexity. Estimating based on time will only give you an aneurysm. And you’ll always be wrong.

Honestly, the most agile places I’ve worked don’t really pay too much attention to actually estimating the timeline for getting much of anything done. Instead, it’s wiser to focus on the most important stuff that goes at the top of the priority list, and work on it until it's done. Then move on to the most important next thing. If it’s important, it doesn’t matter how long it takes. Granted, this takes continuous delivery to the ultimate extreme and really is a Kanban-only approach.

Keep It Simple

After all, this is Simple Programmer, right? The most important thing to do when you start making adjustments in your workflow is simplification. Start with a simple board that has three statuses on it: To Do, In Progress, Done. Hopefully, that workflow is painfully obvious, but if not:

To Do: You (or your team) haven’t started on these things yet. The most important stuff is at the top.

• In Progress: You (or your team) are working on these tasks actively moving them towards done.
• Done: It’s in production and doing what it’s supposed to do.

The concept of software really does boil down to just one thing: getting it deployed. There’s tons of methods for getting it there, but they all end up in the same place: code that’s in an executable or website of some kind.

If you wanted to get really simple, do what some of my project management focused colleagues recommend, and start analogue. Find a big blank wall, get some sticky notes or some index cards that you can tape to the wall, and physically move the cards around. Visibility is obviously key because you want everyone to know what everyone else is working on.

Also, check in with one another for a quick session to kick off (or close out) each work day. Look at wherever your board is, talk about the issues on it, and talk about anything holding up progress. Then get to work. The whole purpose of this check-in should answer:

• “What did I do yesterday (today)?”
• “What am I doing today (tomorrow)?”
• “Is anything in my way?”

Additionally, think about one simple concept that really makes sense: if it hurts, do it more often, as frequently as possible. Keep doing it until it stops hurting. This can be anything. Test code, deployment, writing a spec, and so on and so forth. Find the stuff that really annoys the team, and keep doing it and optimizing it until it doesn’t hurt.

One Backlog to Rule Them All

More than any other thing you can do to make an agile project work well is a prioritized and organized backlog. Make sure that the whole team is working from the same work queue. How you decide to divvy up the work and choose the most important tasks depends on the dynamics of your team. But make sure there’s only one list.

Finally, make sure everyone on the team, and in the entire company, knows exactly what’s on there and why it’s there. If everyone you work with and around knows what everyone else is doing and why it’s a priority, then the rest of the process can improve over time—and you’ll always have plenty of time to improve it.

Moving Along The Agility Spectrum

Regardless of the work situation you find yourself in, you’ll want to take steps to move towards a more agile approach. Granted, not everyone thinks agile approaches are the right approaches. Kanban style development pushes a team as close as possible to true continuous delivery. Realistically, this doesn’t always jive with organizational direction.

What do you think? Can we fix processes that don’t concentrate on flexibility and agility? Or is the software engineering profession really leaving rigidity in the dust?