In my last post, I detailed four of the biggest reasons why software developers suck at estimation, but I didn't talk about how to solve any of the problems I presented.
While estimation will always be inherently difficult for software developers, all hope is not lost.
In this post, I am going to give you five real tips you can utilize to become better at estimation–even for complex software development tasks.
Tip 1: Break Things Down Smaller
In my last post, I talked about how lengthy time periods, that are so common with software development projects, tend to make estimation very difficult and inaccurate.
If you are asked to estimate something that will take you five minutes, you are much more likely to be accurate than if you are asked to estimate something that will take you five months.
So, how can we solve this problem?
There is actually a relatively simple fix: Break things down into smaller chunks and estimate those smaller chunks.
Yes, I know this seems simple and obvious–and I know that this approach is often met with skepticism. There are plenty of excuses you can make about why you can't break things down into smaller pieces, but the truth is, most things can be broken down–if you are willing to put forth the effort.
I've actually talked about why smaller is better and how to break down a backlog in the past, so I won't rehash all the details again here.
The key point to realize is that you are never likely to get good at estimating large things. Well, let me rephrase that: The only way you are going to get good at estimating large things is to be learning how to break them down into many smaller things.
If you really need to accurately estimate something, it is well worth the effort to spend the time breaking down what you are estimating into much smaller pieces.
For example, suppose I was going to estimate how long it will take me to write a blog post. It's not a very large task, but it's big enough that estimates can be a bit inaccurate.
If I want to be more accurate, I can break down the task into smaller pieces.
Consider the difference between trying to estimate:
- Write and publish a blog post
And:
- Research blog post and brainstorm
- Outline blog post
- Write first draft of blog post
- Add images, links and call-outs
- Schedule post for publishing
By breaking things down into smaller pieces, I can more accurately estimate each piece. In fact–here is a little trick–when things are broken down this small, I can actually time-box certain parts of the process–which is effectively ensuring my estimate is accurate (but, we are jumping ahead, we'll talk more about time-boxing in a little bit.)
The next time you are asked to implement some feature, instead of estimating how long you think it will take you to do it as a whole, try breaking down the task into very small pieces and estimating each piece individually. You can always add up the smaller estimates to give a more accurate estimate of the whole.
But wait! I know exactly what you are going to say is wrong with this kind of estimation. Sure, each individual piece's estimation may be more accurate, but when you add them back together, in aggregate, you'll still get the same level of error as you would from estimating one large thing.
All I can say to that argument is “try it.” To some degree you are right, the smaller errors in the smaller pieces will add up and cause the whole to be off by more in aggregate, but the smaller pieces also tend to average out. So, some take less time and some take more, which means that overall, you end up a lot more accurate than estimating one large thing with a large margin of error.
Tip 2: Taking time to research
Why do you suck at estimation?
Because you don't know enough about what you are estimating.
In the previous post, I talked about how the unknown unknowns, that plague many software development projects, make estimation extremely difficult, but I didn't really talk about how to deal with these things that we don't know that we don't know.
Again, the answer is really quite simple: research.
The best way to get rid of an unknown unknown is to know about it.
Whenever you are tasked with estimating something, your first instinct should be to want to do some research–to try and discover what it is that you don't know that you don't know yet.
Unfortunately, most software developers don't immediately think about doing research when trying to estimate something. Instead, they rely on past experience. If they've done something in the past that they deem similar enough, they will confidently estimate it–ignoring the possible pitfalls of the unknown unknowns. If they've never done something similar to what they are being asked to estimate, they'll assume there are unknown unknowns everywhere and come up with estimates full of padding.
Neither approach is good. Instead, you should first try and estimate how long it will take you to research a task before giving an estimate of how long the actual task will take. I've found that most software developers are pretty good at estimating how long it will take to research a task, even though they may be very bad at estimating how long it will take to complete the task itself.
Once you are armed with research, you should have fewer unknown unknowns to deal with. You may still have some unknowns, but at least you'll know about them.
But, how does this look in reality?
How do you actually research tasks that you are supposed to be estimating?
Well, sometimes it involves pushing back and planning things out a bit ahead of time. I'll give you an example of how this might work on a Scrum or Agile team.
Suppose you want to start improving your estimates by doing research before estimating tasks. The problem is that when you are working on an Agile project, you usually need to estimate the tasks in an iteration and don't really have the time to research each and every task before you estimate it–especially the big ones.
I've found the best thing to do in this scenario is instead of estimating the big tasks right up front, to push the tasks back one iteration and instead estimate how long it will take to research each big tasks.
So, you might have in your iteration any number of small research tasks which only have the purpose of getting you enough information to have a more accurate estimate for the big task in the next iteration. During these research tasks, you can also break down large tasks into smaller ones as you know more about them.
Wait… wait.. wait… I know what you are thinking. I can't just push a task into the next iteration. My boss and the business people will not like that. They want it done this iteration.
Right you are, so how do you deal with this problem?
Simple. You just start planning the bigger tasks one iteration in advance of when they need to be done. If you are working on an Agile team, you should adopt the habit of looking ahead and picking up research tasks for large tasks that will be coming up in future iterations.
By always looking forward and doing research before estimating anything substantial, you'll get into the habit of producing much more accurate estimates.
This technique can also be applied to smaller tasks, by taking, sometimes, just five or ten minutes to do a minor amount of research on a task, before giving an estimation.
The next time you are trying to estimate a task, devote some time upfront to doing some research. You'll be amazed at how much more accurate your estimates become.
Tip 3: Track your time
One of the big problems we have with estimating things is that we don't have an accurate sense of time. My memory of how long past projects took tends to be skewed based on factors like how much I was enjoying the work and how hungry I was.
This skewed time in our heads can result in some pretty faulty estimations.
For this reason it is important to track that actual time things take you.
It is a very good idea to get into the habit of always tracking your time on whatever task you are doing. Right now, as I am writing this blog post, my Pomodoro timer is ticking down, tracking my time, so that I'll have a better idea of how long blog posts take me to write. I'll also have an idea if I am spending too much time on part of the process.
Once you get into the habit of tracking your time, you'll have a better idea of how long things actually take you and where you are spending your time.
It's crazy to think that you'll be good at estimating things that haven't happened yet, if you can't even accurately say how long things that have happened took.
Seriously, think about that for a minute. No, really. I want you to think about how absurd it is to believe that you can be good at estimating anything when you don't have an accurate idea of how long past things you have done have taken.
Many people argue that software development is unlike other work and it can't be accurately estimated. While, I agree that software development is more difficult to estimate than installing carpets or re-roofing houses, I think that many software developer's suck at estimation because they have no idea how long things actually take.
Do yourself a favor and start tracking your time. There are a ton of good tools for doing this, like:
If you are curious about how I track my time and plan my week, check out this video I did explaining the process I developed:
By the way, following this process has caused me to become extremely good at estimating. I can usually estimate an entire week worth of work within one-to-two hours of accuracy. And I know this for a fact, because I track it.
Tip 4: Time-box things
I said I'd get back to this one, and here it is.
One big secret to becoming a software developer who is better at estimating tasks is to time-box those tasks. It's almost like cheating.
When you time-box a task, you ensure it will take exactly as long as you have planned for it to take.
You might think that most software development tasks can't be time-boxed, but you are wrong. I use the technique very frequently, and I have found that many tasks we do tend to be quite variable in the time it takes us to do them.
I've found that if you give a certain amount of time to a task–and only that amount of time–you can work in a way to make sure the work gets done in that amount of time.
Consider the example of writing unit tests:
For most software developers, writing unit tests is a very subjective thing. Unless you are going for 100% code coverage, you usually just write unit tests until you feel that you have adequately tested the code you are trying to test. (If you do test driven development, TDD, that might not be true either.)
If you set a time-box for how long you are going to spend on writing unit tests, you can force yourself to work on the most important unit tests first and operate on the 80 / 20 principle to ensure you are getting the biggest bang for your buck.
For many tasks, you can end up spending hours of extra time working on minor details that don't really make that much of a difference. Time-boxing forces you to work on what is important first and to avoid doing things like premature optimization or obsessively worrying about variable names.
Sure, sometimes, you'll have to run over the time-box you set for a task, but many times, you'll find that you actually got done what needed to be done and you can always come back and gold-plate things later if there is time for it.
Again, just like tracking your time, time-boxing is a habit you have to develop, but once you get used to it, you'll be able to use it as a cheat to become more accurate at estimates than you ever imagined possible.
Tip 5: Revise your estimates
Here is a little secret: You don't have to get it right on the first go. Instead, you can actually revise your estimates as you progress through a task.
Yes, I know that your boss wants you to give an accurate estimate right now, not as you get closer to being done, but you can always give you best estimate right now and revise it as you progress through the task.
I can't image any situation where giving more up-to-date information is not appreciated.
Use the other four tips to make sure your original estimate is as accurate as possible, but every so often, you should take a moment to reevaluate what the actual current estimate is.
Think about it this way: You know when you download a file and it tells you how long it will take? Would you prefer that it calculated that duration just at the beginning of the download process and never updated? Of course not. Instead, most download managers show a constantly updated estimate of how much time is left.
Just going through this process can make you better at estimations in general. When you constantly are updating and revising your estimates, you are forced to face the reasons why your original estimates were off.
What about you?
These are just a few of the most useful tips that I use to improve the accuracy of my estimates, but what about you? Is there something I am leaving out here? Let me know in the comments below.
Also, if you liked this post, join over 6,000 other software developers that are part of the Simple Programmer community. Just sign up here and start your journey to becoming a better, more fit, and productive software developer.