The goal of software development is to solve problems.
At its heart, software development is really about solving problems through automation.
Many times we have the tendency to make software development about creating solutions. It may seem that these are the same thing, but there is a subtle difference.
The difference is the focus
When we are trying to solve problems, we are obsessed with the question of why. It is only by understanding why that we can know what to build.
When we are trying to build a solution, we are obsessed with the question of what. We want to know what to build so that we can build it.
This is fairly natural for software developers since the what is something we can control. Any fairly skilled software developer can build any what you can describe to them. Just like any skilled carpenter can build any type of wooden furniture you desire.
The difference is that software development is about more than just building things. A carpenter doesn’t have to focus so much on the why, to build you a piece of furniture. But a truly skilled carpenter will ask you why. A skilled carpenter might make a house call to the project to custom build an entertainment center suited to your needs.
A truly skilled craftsman of any craft, needs to know why.
It is odd that we constantly seem to neglect the why when building software (at least I do), primarily because we think we don’t have time for the why, when in the end solving the why is the only thing that really matters.
Do you really want a carpenter to build you a perfectly crafted entertainment center with plenty of custom shelves for all your music CDs when you don’t own any music CDs?
Why focusing on what doesn’t work
Consider for a moment what would happen if your GPS system stopped showing your route and the directions you were going to take ahead of time, but instead abruptly told you to turn right when you were supposed to turn.
One of the early GPS systems I used for navigation did exactly this. It was very frustrating! I would constantly shout at it that telling me what to do exactly when I am supposed to do it wasn’t any help. I needed some forewarning so I could switch lanes and prepare to turn.
Also, have you ever asked your GPS the question “why are you taking me this way?”
Focusing only on the what results in split-second decisions and avoidable mistakes. When you are focusing on the what you are not thinking, just doing.
Contrast this with a good GPS navigation system.
What makes a good navigation system good?
I’ve found that good systems will alert me far in advance to what my next move is. This gives me time to switch lanes or to have a better understanding of the bigger picture of the trip.
Now this allegory won’t carry us very far in the software development world. You don’t really need to know why your GPS system is taking you a particular route in order to get there, but it does demonstrate how focusing only on the immediate what can lead you astray.
Perhaps a more direct analogy related to software development is outsourcing.
If you’ve ever worked on a software development project that has had a portion of its work outsourced, you may have felt the pain of what happens when oftentimes perfectly competent programmers focus completely on the what and might not even have the slightest clue about the why.
I’m not trying to knock outsourcing, and I’m not even specifically talking about outsourcing to different countries. (The same problems exist in outsourcing whatever the conditions are.)
Oftentimes outsourced projects are treated like a set of blueprints that need to be built. The poor developers working on the outsourced project tend to get thrown under the bus when they build something based off just the what and can’t anticipate the why.
Why focusing on why is better
Focusing on the why is focusing on providing holistic cures to software development problems rather than treating each symptom individually.
Just like you’d like to have a doctor set a broken leg and put it in a cast so that it can be wholly cured, rather than give you some pain medicine to manage the pain, give you crutches to help you walk, and send you to therapy where you can learn to live without the use of your leg, your customers would probably rather you built something that actually is focused on solving their problem, not something that makes their symptoms easier to live with.
If we start building software from the what that is given to us, versus the why, we are at a distinct disadvantage because we only have the ability to treat the symptoms of a problem that we don’t even understand.
Focusing on the why is vitally important because it helps us to better think about and design the what.
Reuse is also much more likely to be present on why focused solutions than what focused ones.
Which software component is more likely to be adopted for reuse: a component that addresses a specific set of account balancing steps or a component that addresses the general problem your customer has of doing accounting?
Why focused solutions are much more likely to be shared across many customers, while what focused solutions are often very specific to a single customer.
Let’s not forget one of the most important reasons why it is important to focus on the why.
Missing the why is costly!
You can build the what someone is asking for, and you can do it perfectly, but at the end of the day if what you built doesn’t solve the problem (the why,) it’s going to have to be redone or thrown out completely.
You may think you are saving time by starting with the what, and you may actually build your solution faster by not taking time for everyone on the project to understand the why, but chances are you’ll build the wrong thing, and when you do, you’ll lose any time benefit you may have accrued.
It is very hard for a team to focus on the what. Focusing on the what tends towards segregating the responsibility of the team. The team members end up only having ownership for their own parts of the solution, rather than solving the problem at whole. So, when a team that focuses on the what ends up with a problem, finger pointing a shirking of responsibility inevitably ensue.
By instead focusing the team on the why, every member of that team becomes responsible for solving the problem rather than solving their part of the problem.
The most compelling reason to focus on the why?
For many problems, just having a thorough understanding of the problem takes you 90% of the way to the answer. I have spent countless hours trying to design and architect a clever solution to a problem without having a really good grasp of the problem that I’m trying to solve, only to find that 20 minutes spent actually fully understanding the why of the problem resulted in the answer becoming completely apparent.
How to know if your focusing on what
I’ve got a pretty good idea of how to know when I am focusing on what instead of why. Not because of any special wisdom on my part. No, it is because I fall down in this area quite a bit myself.
I’ve learned the hard way, and I am still learning how to identify when my focus needs to shift from what to why.
Here are some simple questions that you may find helpful to evaluate your own situation. I’d be glad to hear any others that you can can think of or use routinely yourself.
- Do you’re backlogs, stories or general work items state a solution instead of a problem?
- Do you know how to actually use the thing you are building? (Whoops, I am so guilty of this one so many times.)
- Can you define in simple terms the problem you are trying to solve? (Einstein said “If you can’t explain it simply, you don’t understand it well enough.”)
- Can you clearly define why the what your are building is needed and what purpose it will solve?
- Can you walk through the manual steps of the process you are automating, understanding each step and why it is necessary?
Taking the time to focus on the why instead of jumping into the what can be difficult to get accustomed to, but it is worth the effort. You’ll find much more satisfaction in your work when you understand the purpose it is going to serve, the true problem you have solved. In addition your customers will thank you!
Deploying software doesn’t have to be that complicated!
I’ve seen and built many software building and deployment solutions over my career, and I have come to find that most software deployment can be boiled down to a simple process.
I’m not trying to give you a solution for your software deployment automation, nor am I trying to perfectly model your exact process.
What I am trying to do in this post, is to help you to simplify your process.
If you can identify the parts of your deployment process that fit into the simple steps I am going to outline below, it should be much easier for you to automate your deployment process.
Even though software build processes, infrastructure and components are unique, I have found that most software deployment processes can be simplified into the following steps.
- Build software without configuration
- Create environment specific configuration.
- Create a set of database changes.
- Bundle software, configuration and database changes.
- Apply new software
- Apply new configuration
- Apply new database changes
- Start it back up
You might read through these steps and think “well duh.”
You might be tempted to say “my process is more complicated than that.”
I’m not going to argue with you. You are right, your process is probably more complicated than that. But, does it need to be?
Can you simplify your process to fit into these steps?
Sure, the implementation of these steps is likely to be fairly complex and vary for each type of software, but if you can distill the process into these steps, you can much more easily automate that process.
Where people go wrong
The big key to my simple version of deployment is
Build software without configuration
You MUST do this! Departing from this step causes all kinds of pain and complexity. Please don’t try to build your software and the configuration for an environment at the same time. These things must be pulled out from the get g or you will have the pain of trying to tease them apart later – or you will have to create separate builds for each environment.
It is also critical that the same bits that were built by your build server are what is deployed to each environment!
I will say that this isn’t the easiest problem to solve. You may need to have a separate build process that builds up the configuration for an environment.
Separating the two will also force you down the path of building a process to apply that configuration to an environment.
But, if you are willing to accept that this is just a must and bite through this pain, you’ll come out on the other side clean (even though you had to crawl through tunnels of crap.)
The whole story
Now that I’ve hopefully convinced you to separate your configuration from the building of your software, let’s go over the big picture of a deployment using the simple process outlined above.
It all starts out when you build your software. Perhaps you have a continuous integration build server setup that is automatically building software on each check-in; perhaps you are manually kicking off a script.
Once you have built your software, you have some bits that you should be able to apply to any environment. Nothing that you built here should be machine or environment specific in any way.
Now, you kick off another process, or perhaps one was kicked off simultaneously by your continuous integration server. This builds up the configuration for the environment you are going to deploy to.
A similar process is kicked off—also could be simultaneous, for generating a list of database changes that need to be applied to the target environment.
Now that you have your bits, configuration and database changes, you are ready to deploy.
If you are smart, you’ve even built these ahead of time and they are just waiting for when you need them.
Next, gather up the artifacts and move them to the deployment target where you actually apply them.
First, unpack your bits and put the new bits into place. (You may or may not need to take your application fully offline to do this.)
Then apply the new configuration on top of your newly installed bits for that environment.
Finally, apply database changes for that environment.
Now you should be completely deployed and can start up your application.
But how do I do it?
Perhaps you agree with me that the actual process should be what I have outlined and described, but now you are at the point of implementing a solution.
How do you actually automate this stuff?
Good question. If you figure out a simple answer, let me know.
This is the point where you might be writing custom tools and scripts to get all this working. The key is to take it one step at a time.
There are at least two tools out there that I know of that help you do this. I can’t speak for either of these tools, since I haven’t used them myself, but I have heard good things about them.
One other thing to consider is how you are going to get the right stuff to the right server. You will want to think about things like:
- Promoting build products
- Preloading promoted products to servers to make deployment faster
- Getting through firewalls by having the software or some other process PULL the upgrade to your target, rather than you PUSHING it there.
- Rollback, or some kind of mitigation strategy if things go wrong. (My recommendation here is not to get fancy. I have NEVER seen a successful rollback, only a database restore followed by a manual code restore. If you mess up bad, just count on restoring the machine and the database.)