I already know your biggest problem.
Because, I have that same problem and so do millions of other people across the globe.
To quote the internets, “Ain’t nobody got time for that.”
I don’t know a more true statement in all of history.
It seems that no matter how much time we have, it’s never enough.
It seems the perfect excuse for any situation is simply that we don’t have time.
But, it’s not true. We actually have enough time. In fact, in most cases, we have plenty of time. We just have trouble realizing it.
Why we think we don’t have enough time
There are two main reasons why we think we don’t have enough time.
First, we waste a large amount of time. And, second, we don’t use the time we have wisely.
To sum it up, we procrastinate and watch too much TV.
This may seem like a joke, but I’m dead serious.
If you think you don’t have enough time, it’s probably because you are wasting a big chunk of your day each day doing something completely unproductive, like watching TV, or it’s because when you do sit down to work, you spend more time procrastinating instead of actually getting what you need to do done.
I’ve ranted before on the dangers of watching TV. I don’t watch any TV myself and very rarely even watch a movie. But, it’s not just TV that is the problem. I pick on TV, because it is easy to pick on. Most people would get back two-to-four hours a day if they just quit watching TV—more than enough time to do whatever it is you claim to not have enough time for.
The real problem is anything that is taking your time, but not giving you anything in return. I’m not saying that you should spend all your time working, but what I am saying is that whatever activity you are doing with your time, you should be getting some tangible benefit from it.
But TV relaxes me, you say. Really? Does it? I mean yes, it feels good to plop down in front of the TV and laugh at some silly sitcom character’s latest plight, but does it really relax you so much that you get more time out of it than what you put into it?
What I mean by this is that if you stopped watching TV, and instead did something else with your time, do you honestly think you’d be less productive at everything else you did, because you were not as “relaxed?”
I keep picking on TV, but maybe your waste of time is playing World of Warcraft online, playing on social networks, reading the news, or even spending an inordinate amount of time going through some ritual to get ready for work in the morning.
Figuring out where your time is going
Your first step on your quest to reclaiming your time is to track it.
Look, I’m going to be honest with you here. This part isn’t going to be fun and you are not going to want to do it correctly, but if you really want to know where your time is going, you are going to have to start tracking it—at least temporarily.
Here is what I want you to do:
First, make a sketch of where you think your time is going. Pull out a piece of paper, or print out a sheet from a daily time planner and chart exactly where you think you spend your time during the day.
Make this chart pretty detailed. If you think you get up at 7:00 AM, put that down. Put down how long it takes you to get ready for work and what time you are out the door. Put how long it takes you to “settle in.” Put how long you work. How long you take for lunch. Put it all down and chart it on a 24 hour timeline.
Now, just looking at this “sketch,” you’ll probably already have a good idea of where you are wasting your time. But, here is the clincher: It’s not even accurate.
So, here is what you do next:
After you have your chart of how you think you spend your day, get another blank schedule and carry it around with you for at least three days. Plot exactly how you spend your day.
Yes, this is tedious. Yes, you won’t want to do it, but do it anyway. Track exactly how long it takes you to eat lunch—just don’t show your boss. Track exactly how long it takes you to “settle in.” (Again, you might not want to show your boss that one either.) And, don’t forget that TV time.
Ok, so once you’ve done this you’ll have two things:
- How you think you spend your time
- How you actually spend your time
Now, the only thing you are missing is…
How you actually want to spend your time
The next, step—and you may have guessed it already—is to chart how you’d actually like to spend your time each day. What is your ideal schedule?
Using the information about how you are actually spending your day and how you thought you were spending your day, you should be able to come up with an ideal division of your time for a typical day.
You might end up having a few different schedules for different days of the week, but this is the first step to actually taking back control of your time—and your life.
Think about it this way: You probably know where you are spending your money. Well, let me take that back. If you are financially responsible, you know where you are spending your money. If you aren’t financially responsible, you at least have to make a conscious effort to spend money—it doesn’t happen automatically. You usually have to pull out a card or some cash.
Spending time though, seems to have no cost associated with it. If you feel like you don’t have enough time, it’s not because you are actually living in time-proverty, it’s because you are not actually aware of how you are spending it. You need a time budget.
Now, let me say this right off the bat: You are not going to perfectly live your pre-planned schedule. That is ok. You don’t have to, you just have to be close.
You aren’t going to be able to change everything overnight. The reason you spend your time like you do is because of the ingrained habit that you’ve formed over years or even decades. It takes time to develop new habits to replace old ones.
So, you are going to have to go through a bit of a transition period, while you recalibrate your schedule from the default way you are currently spending—wasting—your time, to the new budgeted schedule of how you want to spend your time.
You’ll probably find that you actually have time to do the things you claimed you didn’t have time to do—now that you’ve actually planned them into your schedule.
If you still find that you don’t have enough time to do what you want to do, you’ll really have to question whether or not you really want to do it, or if you are even trying to do too much.
It’s all about priorities
The whole point of this exercise—of this blog post—is to make you realize that time management is all about priorities.
What I mean to say by this is not that your priorities determine how you spend your time, but rather that how you spend your time shows you what your priorities currently are.
You have to understand your current priorities in order to create new ones or to lower the value of current ones.
By tracking how you are actually spending your time, you are learning about your current priorities.
By projecting how you think you are spending your time, you are indicating what you think your current priorities are.
By actually scheduling your time, you are setting your priorities.
So, if you do all this and still don’t have enough time to learn how to cook, get your degree, get a better job, start that side project, or even start a blog, it’s not really because you don’t have enough time, it’s because you choose to spend your time somewhere else.
(Also, if you are interested in more about this topic, I recommend the book: Eat That Frog!: 21 Great Ways to Stop Procrastinating and Get More Done in Less Time)
But, didn’t you say something about not using time wisely?
So, we’ve only really solved half of the problem here.
Remember how I said there were two main problems? That not only were you wasting a large amount of time, but you also weren’t using that time wisely?
We’ve already covered wasting time. If you are allocating the time in your day how you would like to allocate it, time is no longer being wasted. You are using your time exactly how you want to be using it.
To be clear, this could include watching TV or playing video games. As long as you have made a conscious decision of how you want to be spending your time and that is how you are doing it, you aren’t wasting that time at all. (You might not be meeting your objectives in life, or accomplishing what you want to accomplish, but at least you are making the conscious choice to do so—not blaming it on the lame excuse of not having enough time.)
Now, about using time wisely.
This really boils down to productivity and procrastination.
If you’ve already budgeted your time, the only other thing you can do to get back more time is to reduce the time budgeted things take.
If you lollygag your way through getting ready for work in the morning and something that should take you one hour takes you two hours, you are not using your time wisely. I won’t say wasted, because you planned out that time, so it’s not truly wasted, but you could have accomplished the same goal in a lot less time.
The same goes with “settling in for work,” or even getting a task done at work. The more you procrastinate and choose to use your time in an unwise manor, the longer you have to allocate for the various things you are doing.
But I can’t cut down my nine-to-five, so why bother
Now, you may say, well John, that is nice and all, but I have to be at my desk from 9:00 AM until 5:00 PM, what difference does it make if I use that time “unwisely” or not?
It might not make that much of a difference to your paycheck, but it might make a whole lot of difference to your bottom line and your career.
If you can accomplish more work and learn more things during your nine-to-five, over time this is going to add up to a huge benefit.
Besides, not everything falls into that nine-to-five category, and even though most employers would probably frown on it, wouldn’t it be better to spend time working on some side-project or personal development project than browsing Facebook?
Let’s be honest here. We all waste time at work. At the very least we could waste that time in a productive manner.
Getting serious about lost time
I get most of my blog posts done in under two hours. It literally takes me less than two actual hours from when I sit down to write the blog post until it is scheduled and ready to go.
That might seem like a lot of time, but considering that many of my blog posts are around 2,000-3,000 words and contain a bunch of images and links, that is a pretty short amount of time.
The secret to how I get my blog posts done so quickly each week is that when I sit down to write a blog post, I sit down and actually write it. I don’t sit and ponder the subject for an hour, forget what I was doing and check Facebook or my email. I don’t write a little and then go grab a snack. I sit down and I write until I am done.
I make an efficient use of my time.
You should apply this kind of work-ethic to everything you are doing if you really want to maximize your time.
It’s not for the faint of heart though. When I first started using the Pomodoro technique and actually focusing on making the most of my time and not procrastinating, it took a lot out of me. I did not expect how exhausting it would be.
For a detailed account of how I plan my week, check out my YouTube video.
And to learn more about how I use the Pomodoro technique, you might want to check out my book “Soft Skills: The Developer’s life manual.”
The point is: there is a lot of hidden lost time in the tasks that you are doing that you aren’t doing very efficiently. If it took me four hours to write a blog post instead of two, I’d get a lot less done each day. I’ve have to choose some things I couldn’t get done.
Summing it all up
Ok, so to sum it all up, if you think you don’t have enough time and you want to get back more of your time, here is what you need to do:
First, go through the process of charting how you think you spend your time, how you actually spend your time and how you want to spend your time.
Once you do this, you’ll shift your priorities and hopefully get rid of some big time wasters that are taking up time you’d rather be using somewhere else.
This should be the single biggest way you find more time. It may involve dropping some things that turn out not to be that important, but at least you’ll be making a conscious choice.
If you still don’t have enough time, the only other place to squeeze time from is tasks you are already doing.
If you can do things more efficiently and use your time more wisely, you’ll get a lot more done and have a lot more time on your hands.
Try to find one or two things you do in your day that take up a large amount of time and see if you can cut them in half or at least reduce them by a third.
All the bonus time you get from implementing these techniques you can allocate to other things you want to be doing, but just haven’t had enough time for.
It’s time to stop using that old excuse of not having enough time and to actually do something and make it so you do.
If you liked this post and would like to hear more of what I have to say, join the Simple Progammer community and get blog posts like these, inspirational videos, tutorials and more directly in your inbox—all for free. I won’t spam you, but I will try and motivate you.
Backing up your data is really important.
We’ve all heard too many stories of hard drives crashing or computers getting lost or stolen without having a backup and their owner’s suffering a horrible loss of irreplaceable data.
So, if we all know that backing up data is so important, why don’t we do it?
Well, some of us do, but I know that a majority of software developers I talk to are either not really doing good backups at all or are doing what I would call a half-ass job.
The reason for this is simple: Coming up with a good backup solution is difficult–or at least it can appear that way.
That’s why I am writing this blog post. I want to make it as simple as possible.
I’m going to show you a simple approach to create a backup strategy–not just a solution–that you can easily implement.
The basic approach will be as follows:
- Reduce the amount of “stuff” that needs to be backed up
- Divide backed up data into two categories: critical and non-critical
- Have three copies of data, two local and one offsite
- Make everything automated
Step 1: Reducing the amount of “stuff” that needs to be backed up
The easiest way to simplify your backup solution is to start by reducing the amount of stuff that you need to backup. The less you have to backup, the easier it will be to manage those backups and to actually do the backups themselves.
What we want to do is go through all the data that we think we need to backup and try to get rid of as much of it as possible.
Most people I know, especially software developers and IT people, are storing all kinds of stuff that they will never need.
I used to rip all kinds of movies, video games and music to my computer and save them in a huge library so that I would have access to all this stuff digitally if I ever needed it.
Guess how often I actually needed some random movie I already watched, a video game I already beat or a book I already read?
Just about never.
Now, I realize that some people actually do use their huge libraries of media. For example, kid movies often get watched multiple times, but you have to admit that there is probably a lot of stuff that you will never ever touch again.
I tried to purge as much stuff that I know I will never likely touch again as possible.
I know you might be resistant to doing this, but let me try and convince you that this is a good idea, then you can ignore me and back it all up if you want to.
First of all, think about how easy it is to rent digital content on demand today or buy something off of Amazon or Ebay. Do you really need to store a copy of “A Night at the Roxbury?” Probably not, if you ever want to watch it again, just pay a few bucks and get it here.
If you watch a lot of movies, you’ll be much better off having a subscription to Netflix than you will trying to store a copy of every movie you’ve ever had your hands on. Think about how many hours you are wasting ripping movies to disk and meticulously organizing them. How many of those movies do you actually watch?
The same goes for music, books and video games. Most things only get consumed once. Get rid of as much of this stuff as possible. You’ll not only stop wasting time storing all this stuff, but your backups will be easier and you will find that a huge mental load is lifted from you.
Even huge music collections are mostly a waste of time. There are multiple music services you can subscribe to that will give you access to just about any music you want for a low monthly fee.
Plus, this trend is only going to increase. More and more stuff is going to be available from the cloud, on demand, for a small rental fee or monthly charge.
Stop saving all that crap.
Step 2: Divide backed up data into two categories: critical and non-critical
Backing up one terabyte of data to the cloud takes a long time and it can be expensive–that is why most people don’t do it.
So, what do you end up with?
Well, if you are like most people, you end up having some kind of local backup and you don’t really have a good cloud or offsite backup in place. It’s just too much trouble to try and backup all that data to the cloud.
If you followed my first step, you should be well on your way to reducing your total data that you need to backup, but we can do much better and get that data to an even smaller amount.
Instead of trying to backup everything to the cloud or offsite, if you focus on backing up just what is critical, you’ll find that it is much more manageable and you won’t need gigabit internet to back everything up.
Take all the data that you want to backup and sort it into two categories: critical and non-critical.
Critical things are things that if you ever lost them, you would be very sad, because they couldn’t be replaced or would cause you some great harm.
A good example of critical data for me is my wife’s photos. If I lost my wife’s photos, I would probably need to find a new wife.
Other critical data for me is current projects I am working on and past projects that I may need to access again at some point in the future.
My Pluralsight courses and other training courses are critical data. My source code for my applications is critical data.
The opposite of critical data is non-critical data–duh.
But what is non-critical data?
It’s data that would suck to lose, but would not be the end of the world. Perhaps data that would be a small inconvenience to you to lose, but could be replaced.
Collections of movies, video games and music fall into this category. Yes, you’ll be disappointed if you lose this data, but you can replace that data even if it might cost you some money.
Now, before you get all uppity about your movie collection, remember, I’m not saying we aren’t going to backup your non-critical data–we will–it’s just that we aren’t going to back this data up to the cloud or offsite.
Other non-critical data might be an image of your computer or development workstation. If you lose that backup, you might have to re-install your operating system or waste some time re-installing other programs, but it won’t be that big of a deal.
Most of your data should be non-critical. Unless, of course, you got rid of a large amount of that data, because you realized the futility of storing digital copies of stuff you won’t ever use again. But, if I haven’t convinced you by now, I probably never will, so we’ll just call your “The Complete Matrix Trilogy (The Matrix / The Matrix Reloaded / The Matrix Revolutions) [Blu-ray]” non-critical data.
Step 3: Have three copies of data, two local and one offsite
Ok, now we are actually ready to back things up.
Most likely, you’ll have a small amount of critical data and either a larger amount of non-critical data or almost none.
The critical data we don’t ever want to lose. So, we need to make sure that there are three copies of that data at all times.
The easiest way to do this is to have:
- one working copy
- one local backup
- and one cloud backup
Today, this is actually quite simple to achieve.
For awhile I was doing this by using a service like CrashPlan. Crash plan allows you to specify folders on a computer to backup to another location and to specify some folders to be backed up to the CrashPlan cloud servers as well.
I was creating two backup sets. One that backed up my critical data to another hard drive in my computer and another backup that backed up to CrashPlan’s servers.
This worked well for a while, but then I realized that I didn’t really need to pay CrashPlan’s monthly fee when I was already paying for extra storage space with Dropbox and also that I wanted to have a central place to backup data locally and not just back up from my one PC. My wife has data she needs to backup and I have a laptop and other devices as well.
Now, don’t get me wrong, CrashPlan is great. I highly recommend it, but if you have a NAS (Network Attached Storage) and an account with either Dropbox or OneDrive, you might not really need to utilize a service like CrashPlan.
But, before I get into the specifics of what I am doing, let’s talk about the strategy one more time.
We want to have three copies of our data. One working copy, one local backup and one offsite backup.
There are many ways to accomplish this; the easiest way is to have a cloud storage solution like Dropbox or OneDrive as an offsite backup and then to figure out some way to have a local backup as well.
The reason why we want an offsite backup and a local backup is so that we are covered in two possible scenarios:
- Your local backup fails and when you go to recover your data you discover this problem. In that case you can just get the data from the cloud.
- Your cloud backup either fails, goes out of business or loses your data. In this case, you can use your local backup to recover your data and move your offsite backup to another service.
If you just put your data in the cloud, it isn’t good enough, because you are relying completely on someone else’s service that you can’t control.
If you just locally back up your data, it isn’t good enough, because you could have a fire and your entire house could burn down, or you could be robbed, or your backup could just fail and you not know it.
How I’m backing up my critical data
So, how am I actually backing up my critical data now that I’ve gotten rid of my CrashPlan subscription?
Well, I invested in a Synology NAS, or network attached storage.
I bought a Synology DiskStation 2-Bay (Diskless) Network Attached Storage (DS213j) which I have attached directly to my network.
It allows any computer in my network to use it as a file server and it runs its own little operating system that can do all kinds of neat things like backup my data to Dropbox or even Amazon Glacier.
There are two big advantages of this kind of device versus having some hard drives in my computer that I am using to create a copy of data:
- The data is accessible by all the computers and devices in my house easily. I don’t have to have my main PC on and connected to the network.
- The Synology devices very easily do a RAID style storage. So, I can have two hard disks in there and if one of them fails, the other one still has all the data.
A distant third, for me, would be that the Synology box can act as a full media server. I don’t use that functionality that much, but I know some people with huge movie collections do.
The big key point for my solution is that the Synology device creates a very good redundant RAID. In my book that counts as two copies of local data.
I just attach my Synology box as a network drive on my computers and devices and I store any data that I want to make sure is backed up there.
Synology has a service you can install that hooks up your device with your Dropbox account (OneDrive support coming really soon.) So, I just share out that Dropbox folder on my Synology drive out to my network and I can drop any files I want backed up in that share and those files will not only be in double backup on the Synology drive, but also backed up to my Dropbox account in the cloud.
My wife can also do the same, so this is very convenient.
I also don’t even worry about having a local copy of a backup on my computer anymore, because I know that the data in the Synology drive is backed up on two hard drives and in the cloud.
How I’m backing up my non-critical data?
What about the non-critical data?
Simple. I just copy that to a share on the Synology device that isn’t backed up to the Dropbox account.
For example, right now, the only thing I really have that I am considering non-critical data is images of my computers. I just put those on a share on the Synology and I don’t worry about them.
Why not just use a backup service like CrashPlan or Mozy?
Again, this whole thing could be done with a backup service like CrashPlan, but if I am already going to have a Dropbox account that I use, I don’t see the point of paying for and managing another backup system.
In fact, I am probably going to switch over to OneDrive exclusively, because Microsoft just recently announced that Office 365 users get unlimited OneDrive storage.
If you want to use a backup service though, go ahead. Just make sure you have a local backup and a cloud based backup for your critical data.
Step 4: Make everything automated (and test it)
If you followed my backup plan or you are using a service like CrashPlan, then you probably don’t need to do much here, because everything is already automated.
CrashPlan automatically backs up the data on your computer as it changes, so you don’t really have to worry there.
And, if you are using a NAS and a service like Dropbox, that is automated as well, because you basically just drop files into the Dropbox folder on your NAS and it automatically syncs with Dropbox.
But, if you are doing something else, just make sure the entire process is automated. You might, for example, want to automate backing up images of your computer. Or you might want to automate getting photos off of your phone or camera and dropping them into a backup location.
Finally, make sure you test everything out.
A backup that has never been tested is worthless.
If you use a service like CrashPlan, try restoring data from it.
If your backup is going to be your Dropbox box and your NAS, test it out. Make sure you can retrieve any data that you need. If you are backing up databases or snap-shotting PCs, make sure you can restore all of those backups, otherwise don’t bother backing them up in the first place.
Nothing is worse than trying to restore a backup and finding out that it was no good or wasn’t working.
Are you backing up your data?
Let me know in the comments below.
And, if you liked this post and found it helpful, join the Simple Programmer community so that I can stay in touch and let you know when I have new posts or other free content you might be interested in.
Also, if you have some other suggestions or think there is something I missed or didn’t consider, leave a comment and let me know.
In my last post, I detailed four of the biggest reasons why software developers suck at estimation, but I didn’t talk about how to solve any of the problems I presented.
While estimation will always be inherently difficult for software developers, all hope is not lost.
In this post, I am going to give you five real tips you can utilize to become better at estimation–even for complex software development tasks.
Tip 1: Break Things Down Smaller
In my last post, I talked about how lengthy time periods, that are so common with software development projects, tend to make estimation very difficult and inaccurate.
If you are asked to estimate something that will take you five minutes, you are much more likely to be accurate than if you are asked to estimate something that will take you five months.
So, how can we solve this problem?
There is actually a relatively simple fix: Break things down into smaller chunks and estimate those smaller chunks.
Yes, I know this seems simple and obvious–and I know that this approach is often met with skepticism. There are plenty of excuses you can make about why you can’t break things down into smaller pieces, but the truth is, most things can be broken down–if you are willing to put forth the effort.
The key point to realize is that you are never likely to get good at estimating large things. Well, let me rephrase that: The only way you are going to get good at estimating large things is to be learning how to break them down into many smaller things.
If you really need to accurately estimate something, it is well worth the effort to spend the time breaking down what you are estimating into much smaller pieces.
For example, suppose I was going to estimate how long it will take me to write a blog post. It’s not a very large task, but it’s big enough that estimates can be a bit inaccurate.
If I want to be more accurate, I can break down the task into smaller pieces.
Consider the difference between trying to estimate:
- Write and publish a blog post
- Research blog post and brainstorm
- Outline blog post
- Write first draft of blog post
- Add images, links and call-outs
- Schedule post for publishing
By breaking things down into smaller pieces, I can more accurately estimate each piece. In fact–here is a little trick–when things are broken down this small, I can actually time-box certain parts of the process–which is effectively ensuring my estimate is accurate (but, we are jumping ahead, we’ll talk more about time-boxing in a little bit.)
The next time you are asked to implement some feature, instead of estimating how long you think it will take you to do it as a whole, try breaking down the task into very small pieces and estimating each piece individually. You can always add up the smaller estimates to give a more accurate estimate of the whole.
But wait! I know exactly what you are going to say is wrong with this kind of estimation. Sure, each individual piece’s estimation may be more accurate, but when you add them back together, in aggregate, you’ll still get the same level of error as you would from estimating one large thing.
All I can say to that argument is “try it.” To some degree you are right, the smaller errors in the smaller pieces will add up and cause the whole to be off by more in aggregate, but the smaller pieces also tend to average out. So, some take less time and some take more, which means that overall, you end up a lot more accurate than estimating one large thing with a large margin of error.
Tip 2: Taking time to research
Why do you suck at estimation?
Because you don’t know enough about what you are estimating.
In the previous post, I talked about how the unknown unknowns, that plague many software development projects, make estimation extremely difficult, but I didn’t really talk about how to deal with these things that we don’t know that we don’t know.
Again, the answer is really quite simple: research.
The best way to get rid of an unknown unknown is to know about it.
Whenever you are tasked with estimating something, your first instinct should be to want to do some research–to try and discover what it is that you don’t know that you don’t know yet.
Unfortunately, most software developers don’t immediately think about doing research when trying to estimate something. Instead, they rely on past experience. If they’ve done something in the past that they deem similar enough, they will confidently estimate it–ignoring the possible pitfalls of the unknown unknowns. If they’ve never done something similar to what they are being asked to estimate, they’ll assume there are unknown unknowns everywhere and come up with estimates full of padding.
Neither approach is good. Instead, you should first try and estimate how long it will take you to research a task before giving an estimate of how long the actual task will take. I’ve found that most software developers are pretty good at estimating how long it will take to research a task, even though they may be very bad at estimating how long it will take to complete the task itself.
Once you are armed with research, you should have fewer unknown unknowns to deal with. You may still have some unknowns, but at least you’ll know about them.
But, how does this look in reality?
How do you actually research tasks that you are supposed to be estimating?
Well, sometimes it involves pushing back and planning things out a bit ahead of time. I’ll give you an example of how this might work on a Scrum or Agile team.
Suppose you want to start improving your estimates by doing research before estimating tasks. The problem is that when you are working on an Agile project, you usually need to estimate the tasks in an iteration and don’t really have the time to research each and every task before you estimate it–especially the big ones.
I’ve found the best thing to do in this scenario is instead of estimating the big tasks right up front, to push the tasks back one iteration and instead estimate how long it will take to research each big tasks.
So, you might have in your iteration any number of small research tasks which only have the purpose of getting you enough information to have a more accurate estimate for the big task in the next iteration. During these research tasks, you can also break down large tasks into smaller ones as you know more about them.
Wait… wait.. wait… I know what you are thinking. I can’t just push a task into the next iteration. My boss and the business people will not like that. They want it done this iteration.
Right you are, so how do you deal with this problem?
Simple. You just start planning the bigger tasks one iteration in advance of when they need to be done. If you are working on an Agile team, you should adopt the habit of looking ahead and picking up research tasks for large tasks that will be coming up in future iterations.
By always looking forward and doing research before estimating anything substantial, you’ll get into the habit of producing much more accurate estimates.
This technique can also be applied to smaller tasks, by taking, sometimes, just five or ten minutes to do a minor amount of research on a task, before giving an estimation.
The next time you are trying to estimate a task, devote some time upfront to doing some research. You’ll be amazed at how much more accurate your estimates become.
Tip 3: Track your time
One of the big problems we have with estimating things is that we don’t have an accurate sense of time. My memory of how long past projects took tends to be skewed based on factors like how much I was enjoying the work and how hungry I was.
This skewed time in our heads can result in some pretty faulty estimations.
For this reason it is important to track that actual time things take you.
It is a very good idea to get into the habit of always tracking your time on whatever task you are doing. Right now, as I am writing this blog post, my Pomodoro timer is ticking down, tracking my time, so that I’ll have a better idea of how long blog posts take me to write. I’ll also have an idea if I am spending too much time on part of the process.
Once you get into the habit of tracking your time, you’ll have a better idea of how long things actually take you and where you are spending your time.
It’s crazy to think that you’ll be good at estimating things that haven’t happened yet, if you can’t even accurately say how long things that have happened took.
Seriously, think about that for a minute. No, really. I want you to think about how absurd it is to believe that you can be good at estimating anything when you don’t have an accurate idea of how long past things you have done have taken.
Many people argue that software development is unlike other work and it can’t be accurately estimated. While, I agree that software development is more difficult to estimate than installing carpets or re-roofing houses, I think that many software developer’s suck at estimation because they have no idea how long things actually take.
Do yourself a favor and start tracking your time. There are a ton of good tools for doing this, like:
If you are curious about how I track my time and plan my week, check out this video I did explaining the process I developed:
By the way, following this process has caused me to become extremely good at estimating. I can usually estimate an entire week worth of work within one-to-two hours of accuracy. And I know this for a fact, because I track it.
Tip 4: Time-box things
I said I’d get back to this one, and here it is.
One big secret to becoming a software developer who is better at estimating tasks is to time-box those tasks. It’s almost like cheating.
When you time-box a task, you ensure it will take exactly as long as you have planned for it to take.
You might think that most software development tasks can’t be time-boxed, but you are wrong. I use the technique very frequently, and I have found that many tasks we do tend to be quite variable in the time it takes us to do them.
I’ve found that if you give a certain amount of time to a task–and only that amount of time–you can work in a way to make sure the work gets done in that amount of time.
Consider the example of writing unit tests:
For most software developers, writing unit tests is a very subjective thing. Unless you are going for 100% code coverage, you usually just write unit tests until you feel that you have adequately tested the code you are trying to test. (If you do test driven development, TDD, that might not be true either.)
If you set a time-box for how long you are going to spend on writing unit tests, you can force yourself to work on the most important unit tests first and operate on the 80 / 20 principle to ensure you are getting the biggest bang for your buck.
For many tasks, you can end up spending hours of extra time working on minor details that don’t really make that much of a difference. Time-boxing forces you to work on what is important first and to avoid doing things like premature optimization or obsessively worrying about variable names.
Sure, sometimes, you’ll have to run over the time-box you set for a task, but many times, you’ll find that you actually got done what needed to be done and you can always come back and gold-plate things later if there is time for it.
Again, just like tracking your time, time-boxing is a habit you have to develop, but once you get used to it, you’ll be able to use it as a cheat to become more accurate at estimates than you ever imagined possible.
Tip 5: Revise your estimates
Here is a little secret: You don’t have to get it right on the first go. Instead, you can actually revise your estimates as you progress through a task.
Yes, I know that your boss wants you to give an accurate estimate right now, not as you get closer to being done, but you can always give you best estimate right now and revise it as you progress through the task.
I can’t image any situation where giving more up-to-date information is not appreciated.
Use the other four tips to make sure your original estimate is as accurate as possible, but every so often, you should take a moment to reevaluate what the actual current estimate is.
Think about it this way: You know when you download a file and it tells you how long it will take? Would you prefer that it calculated that duration just at the beginning of the download process and never updated? Of course not. Instead, most download managers show a constantly updated estimate of how much time is left.
Just going through this process can make you better at estimations in general. When you constantly are updating and revising your estimates, you are forced to face the reasons why your original estimates were off.
What about you?
These are just a few of the most useful tips that I use to improve the accuracy of my estimates, but what about you? Is there something I am leaving out here? Let me know in the comments below.
Also, if you liked this post, join over 6,000 other software developers that are part of the Simple Programmer community. Just sign up here and start your journey to becoming a better, more fit, and productive software developer.
It seems that I am always in a rush.
I find it very difficult to just do what I am doing without thinking about what is coming next or when I’ll be finished with whatever I am working on.
Even as I am sitting and writing this blog post, I’m not really as immersed in the process as I should be. Instead, I am thinking about the fact that I need to get this post done and ready to be published today.
For some reason, I always feel like the clock is ticking down on me. I always feel rushed and that I need to rush things along.
I don’t think I’ve ever sat down and written something that had more than a single rough draft and a final draft. I can’t imagine having multiple drafts of a thing.
Why am I saying this? Because, lately I’ve been meditating on the phrase “anything worth doing is worth doing right.”
Am I really giving it my all?
I’ve been thinking about that phrase a lot and how much I tend to ignore it. I get a lot done, but what I get done isn’t always as satisfying as it should be, because I often find I’m not applying myself as much as I should be.
This “weakness” seems to permeate every area of my life. As I’m running or lifting weights at the gym, I often realize that I’m not giving it my all. As I am writing a blog post, or writing code, I get the same feeling of not giving 100%. When I’m playing with my daughter, or spending time with my family, I’m often not 100% there—but, it’s not like I’m somewhere else either. I’m often just sort of wandering through life a little bit “checked out.”
The best word I can use to describe this is slothfulness. I’ve been feeling this pressing need to eliminate as much slothfulness from my life as possible.
I’m beginning to realize how much time and effort is wasted on doing things in a half-ass manner. If I sit down to do some work and I don’t know exactly what I am doing, if I’m not focused on a specific task I need to get done, I end up wasting a lot of time.
But, it’s actually more than that. I’ve found ways to make sure I am focused on the task at hand in order to make sure that I don’t waste time by taking too long to accomplish a particular task, but what is more difficult is giving 100% to the task at hand. It’s quite possible to be 100% focused, but not to be giving it all you’ve got.
There is a huge penalty in not giving it all you’ve got. This is the real struggle—at least for me—at least right now.
I know the work I am producing could be better. I know the time I’m spending could be more fulfilling, if, I could just fully subscribe to the belief that anything worth doing is worth doing right.
Fixing the problem
The good news is that I have been thinking about some ways to combat this problem. Here are some of my ideas:
First of all, I am going to try and not do anything unless I know what I am going to do and I am going to devote 100% of my focus to that activity.
That doesn’t mean that I have to plan out every aspect of my day ahead of time, but it means that I have to at least plan out what I am going to do before I do it.
For example, today I decided that I was going to go to a coffee shop and get the intro letter for early readers of my book done, write an email that talked about the revisions to the chapters in my book and write this blog post.
I didn’t plan for reading through my email, checking Facebook or doing anything else during that time. I’m sitting here working on exactly what I had planned to work on and I am putting my full focus into that work.
I’ll also plan out when I’ll do certain things so that they aren’t hanging over my head and distracting me from other things I am doing. I find that I can’t focus on the task at hand when I have some uncertainty about another task that I need to get done. Whenever I feel that uncertainty about something that needs to get done, my plan is to schedule it so that I can take it off my mind.
Next up, I’m going to try to have a bit more rigorous standards for what I am doing before I start doing it. I’ve found that it’s often difficult for me to decide what “doing something right” means. It’s pretty subjective and when I feel like I am done with a task, my judgment tends to be skewed. I’m likely to call something done that is “good enough” rather than “right.”
Sometimes the effort to take something from “good enough” to “right” is very small, so it is worth taking the extra time and putting forth the extra effort to go the last mile. I can spend a large amount of time and effort on a task or project and have this gnawing feeling of discontent if I am willing to accept “good enough.” This acceptance of “good enough” often negates the entire reward of the effort, so I want to strive towards doing things right instead of just “good enough,” even if it takes more time.
That brings me to the next point: stop rushing.
I’m always rushing. Always trying to get things done as fast as possible so that I can be as prolific as possible. While being more prolific might have a higher monetary reward, I’ve found it often comes at the cost of feeling discontent with the work being done.
This one is going to be one of the most difficult ones for me. Even now, thinking about this very subject, my fingers are still frantically striking the keyboard as I glance at the clock, worried about how long it is taking me to write this post.
I think a solution to this problem may be to block out ample blocks of time to work on a thing. To purposely give myself breathing room. For example, I might feel less rushed if I came here to write a blog post, that I figured would take me about an hour, but I gave myself two hours to work on it instead. And, if I forced myself to spend the entire allotted time working on it, I would probably not feel rushed and I’d would produce an overall better product.
The next task I do, I am going to try and block off time and force myself to use the entire allotted time.
Living in the moment
Let’s see what else is left. How about living in the moment—another extremely difficult one for me now. I have a difficult time stopping to smell the roses. I imagine that if I stop rushing, I’ll probably solve this problem as well, but for now, I am going to try to start purposely giving 100% to what I am doing at any given time.
With every activity I am doing, work or otherwise, I am going to try and focus 100% on what I am doing and also give 100% to that activity. This one will be difficult—I am sure of it. But, this might be the most important thing to focus on. Sometimes I feel like my life is passing me by because I’m always looking forward or backward—I’m never taking time to stop and smell the roses.
One aspect of this that I have been thinking about is to actively think about what I am doing at any given moment and to clearly define it. For example, if I am sitting on the couch, I should ask myself what am I doing. Am I having a conversation with someone? Am I relaxing? Am I doing something else? At any given time I should be able to define what it is I am doing. It doesn’t necessarily have to be something productive. It is better to actively decide that I am spending time browsing Facebook than it is to just be sitting at a computer “doing nothing.”
In fact, I just purchased The Power of Now: A Guide to Spiritual Enlightenment. Not sure if this book is good or not, but several people have recommended it to me and it came to mind today.
Drop more stuff
Finally, I think I need to drop anything that I am not going to do right. This is the full interpretation of “anything worth doing is worth doing right.” I simply need to make a rule that if I am not willing to do something right, if I am not willing to devote my full energy to it, if I am not willing to slow down and not rush through it, then I simply should not do it at all.
I often have thought that if I stopped splitting my focus so much that I’d be able to be much more successful at the things I do choose to do. This is another difficult one for me, because I tend to see one of my greatest assets as my ability to do so many different things. It’s scary and dangerous to either drop things that I am used to doing or to recommit to them, giving 100% effort.
At a surface level, I know that it would be better to focus on a smaller number of things and to dive deeper into those things, but at a deeper level, I’m scared to do it. I’m the kind of person that likes to leave as many doors open as possible. The thought of closing some doors scares me, but I know I need to do it.
Well, that is about it. I’m trying to use this time in Hawaii, away from my normal schedule to be as introspective as possible. Expect some big changes in the next few months as I start to get everything figured out and set the course for the future.
Yes, that’s right. I am writing a blog post today about Scrum and Scrum Masters.
No, I haven’t lost my mind.
I just realized that out of everything I’ve written about Agile and Scrum, I never talked about what makes a good Scrum Master.
I’ve both been a Scrum Master and I’ve worked on a team with Scrum Masters and from both of those experiences I can tell you that there is much confusion about this particular role on a Scrum or Agile team.
Even the name Scrum Master has confusion around it, is it ScrumMaster or Scrum Master—you can tell, I prefer the latter.
So, let’s talk about why Scrum Masters exist in a Scrum team and what they actually should be doing.
Scrum Masters! What are they good for?
On some teams, unfortunately, it is absolutely nothing.
But, it doesn’t have to be that way. A Scrum Master actually serves a really important role on a properly functioning Scrum Team. (They bring the donuts or bagels in every morning, so the team can get actual work done.)
Ok, I am just kidding about that last part, but in reality it isn’t all that far from the truth. Let me explain.
A Scrum Master really is supposed to be the person who clears the path for the team so they can run as close to full speed as possible. The Scrum Master is sort of like the pit crew for a race car driver.
Without a Scrum Master, a Scrum team is slowed down by impediments which inevitably come up in any development project. It takes time and distracts the team to deal with these impediments, so the whole cadence of the team slows down unless someone external to the development process is moving the boulders out of the way.
So, really, the most important job of the Scrum Master is to remove impediments which may hamper a Scrum team from progressing on backlogs and getting their work done.
This isn’t the same thing as managing a project, because the Scrum Master isn’t deciding how and when things should be done. Instead, the Scrum Master is part of the team and the team as a whole is taking accountability for managing the project.
The Scrum Master also has the role of being the master of the Scrum process—hence the name. This is a tough spot to be in, but is a very important role that many teams neglect. The rules of Scrum are important to a successful Scrum team. One of the reasons why I started to write off Scrum as a process was simply because it was so difficult to get anyone to actually enforce the rules.
This is the job of the Scrum Master; he carries the big Scrum stick and he beats people over the head with it when they step out of line. He doesn’t do this because he is a big power-hungry bully. No, instead, he does this because he knows that the only way the team is going to produce their best work and not waste time arguing over process is if they all follow the process that was agreed upon from the start.
Scrum is intended to be more than just a way to develop software or organize teams, it is also a process that clearly defines what will happen, when it will happen and who will do what.
The Scrum Master is one of the most important roles on the team
It may seem, based on my previous description, that the role of a Scrum Master isn’t all that important to the overall performance of the team, but that is far from the truth.
In reality, the velocity of a team is more influenced by the Scrum Master than any other member of the team—with the exception of that lazy developer that breaks the build all the time and constantly falls asleep at meetings.
Even though the Scrum Master does not have direct control over the management of the team, the Scrum Master’s ability to both remove impediments and enforce the Scrum framework directly affects the team’s ability to get s!@# done.
A poor Scrum Master will let the team flounder and let outside influence distract the team from their work.
A poor Scrum Master will either be too timid or not care enough to force the team to obey the rules of Scrum, causing the whole platoon to go scampering off whatever direction they choose, rifles firing randomly in all directions.
I like to think of the Scrum Master as a guide who takes the team over rough terrain and shows them how to get water from tree leaves on their journey. Sure, the team could manage to bushwhack their way through the jungle without a guide, but it would take them a whole hell of a lot longer to do so—and they’d be much more likely to get eaten by a lion.
So, what should Scrum Masters actually do?
The answer is whatever needs to be done.
You know those gangster movies where some mob boss has a guy they call “the cleaner?” The guy that comes into a sticky situation and can hide a dead body, bribe the right cops, or just make someone disappear? If mobsters were following Scrum, that guy would be the Scrum Master.
The Scrum Master should be part of the team, but not part of the team. The Scrum Master should attend the standup meetings actively trying to spot impediments—especially the ones that aren’t mentioned by the team members, but exist beneath the surface of a problem.
The Scrum Master should ensure that all the Scrum meetings and processes flow smoothly. He should make sure that standups are being used for their correct purpose. He should encourage the team to hold each other accountable and he, himself, should hold the team accountable to what they promised to deliver.
The Scrum Master should be the guy (or gal), who makes things happen. He should know the right people to talk to and know how to get things done. The team should focus on the work, the Scrum Master should focus on the politics. If the team is dealing with politics the Scrum Master has failed.
Most of all, the Scrum Master should be willing to lay it all on the line—to take the hits for the team. Even though the Scrum Master doesn’t control the team and get to boss them around, they are his team and his alone. A good Scrum Master isn’t afraid to take full responsibility for the actions and performance of the team and step in the way of that bullet and take one in the chest if he has to.
Like this post? Sign up for my weekly email and I’ll make sure posts, like this one, are delivered right to your inbox once a week. Also, you’ll get lots of content that I only share with my email subscribers.
In the last year, I:
- Created and produced 30 full length video courses for Pluralsight
- Wrote 56 blog posts
- Produced 40 episodes of the Get Up and CODE podcast
- Created 50 YouTube videos
- Published a book
- Spoke at 4 events
- Billed over 100 hours of contract work
- Created a full product, that I am about ready to launch
- Ran 5 kilometers 3 times a week, every week
- Lifted weights 3 times a week, every week
I’m not saying this to brag–although I am certainly proud of these accomplishments. I am saying these things to prove that I know what I am talking about when it comes to productivity.
Being super productive
Right now–as I type–I have a timer ticking down. The clock shows approximately 14 minutes before I’ll take my next break. I live and die by this clock.
You may have guessed it, but the clock is a Pomodoro timer. For the last year, I’ve been religiously using the Pomodoro technique to not only stay on task, but to plan out my days and weeks.
If you aren’t familiar with the Pomodoro technique, the concept is remarkably simple. So simple, that I first dismissed it as ridiculous. But, thanks to my good friend Josh Earl’s success with it, I decided to give it another try.
You basically set a timer for 25 minutes. During that time you pick a single task to accomplish and work on that task, uninterrupted. After 25 minutes you take a break for 5 minutes and then begin again. After 4 cycles, you take a longer 15 minute break. (There are some variations on this, but that is the basic idea.)
Like I said, it seems pretty simple and unremarkable, but I can’t even begin to express how powerful this technique is for getting things done.
I’m lazy by nature. I have to constantly fight against the side of me that wants to procrastinate and slough off my work. The Pomodoro technique helps keep me focused by forcing me to work uninterrupted for a period of time. It also gives me a measure to compare myself against and realistic targets to aspire to achieve.
My week beings on Monday. On Monday morning I wake up and go to the gym to lift weights. When I get back, I have a protein shake and get to work.
The first thing I do when I get to my desk on Monday is start my Pomodoro timer and open up my “Weekly Plan” Trello board. I use this board to organize my week. It has nine columns. Seven columns for the days of the week, one column for today, and one column for done.
My first task of the day is to create the rest of the tasks that I think I can get done that week. I start off with a checklist of things that I know I need to do every week:
- Blog post
- Podcast episode
- YouTube video
- Newsletter email
- Buffer social network posts
Then, I add cards for the current projects I am working on for that week.
Once I’ve got all the cards I can think of on the board, I start tagging each card with a color that represents how many Pomodoros I think that task will take. I have three categories:
- Green: 1 Pomodoro
- Yellow: 2 Pomodoros
- Orange: 3 Pomodoros
If something is going to take up more than three Pomodoros worth of time, it needs to be split into multiple tasks.
Next up is planning the week. For each day of the week–unless there is something that will take up most of my time–I figure I can get about 10 Pomodoros done. This may not seem like a lot, but believe me, it is. I drag cards into the columns until I have filled up each weekday with 10 Pomodoros worth of cards. For weekends, I usually just drag in about three or four.
My estimates are always on the high side, but they are pretty accurate, because it is fairly easy to estimate based on half hour intervals–especially when many of your tasks are repeated each week. (For example, a blog post is estimated at three Pomodoros.)
I have a similar ritual every single day. The first thing I do, after exercising each day, is to open up my Trello board again and this time plan the day.
New things come up and other things need to get shifted around, so planning for the week alone is not sufficient. Often, I’ll have different tasks that I had vaguely identified at the beginning of the week which I’ll give more clarity to later on.
I first drag things over from the appropriate day into my “Today” column until that column is filled with about 10 Pomodoros worth of work. After that, I’ll take a look at the today columns and think about anything I might have missed that needs to get done that day. Finally, I’ll sort the “Today” column based on priority–I want to make sure I am always working on the most important things first.
Once I’ve got the day sorted out, I go back to the rest of the days in the week and move around cards until everything is balanced again. If I find that I’ve got some empty slots, I’ll create new cards and start filling those slots until I am back at full capacity again.
Once all the prep-work is done, it’s time to actually start working on tasks. I grab the first task off the list, start a timer and get to work. At about 5:00, I stop for the day and add up my Pomodoros. If I didn’t hit at least 10, I count on working a little bit that evening. If I did hit 10, it’s optional.
Why this works
So, you may be wondering why this works–why it is worth even writing about such a simple workflow. Well, even though this workflow seems really simple, there are a few key things going on here that aren’t immediately obvious.
First of all, I am using quotas to make sure that I accomplish the volume of work that I want to produce each week. I have quotas for how many blog posts, podcasts, YouTube videos, and other content that I need to produce each week. The things that are being measured by a quota get dropped onto my board first.
I’m also using a daily and weekly quota when it comes to Pomodoros. Pomodoros are little measurable units of work for me. I know that I should be able to get 10 done each day and that I should be able to get roughly 50 done in a week without killing myself. I know from experience that hitting 60-70 will cause a measurable dip in performance the next week and that if I am doing less than 50, I am slacking off.
Because I have those quotas in place, I know what is expected of me each and every week. I have the power to hold myself accountable to a real measurable standard. I can’t emphasize enough how important this point is. If you don’t have a way to hold yourself accountable to a standard that you want to achieve, human nature will cause you to fall way below the bar.
Another major component that makes this technique successful is the awareness of my capacity within a given amount of time. It is really easy to over or under estimate what you can get done in week, because you don’t normally have a ruler that you can use to measure task duration versus your actual capacity. When I start the week, I know that my capacity is about 50–I’ve got that much gas in my car. I get to choose where I want to drive that car that week–I can only go so far. I have to make a realistic prediction of what I can actually get done. From that prediction I have to prioritize my tasks so that the most important things get done first.
Without this understanding of my capacity, it is easy to fall into the trap of overestimating my ability to get work done and underestimating the take it will take to get the work done. With this system, I have a real metric to compare to. I know that I am not going to get 80 Pomodoros done in a week. I know that in an 8 hour day, I will not get 8 hours of work done. I am eliminating my biases by replacing them with real statistics.
Finally, the dedicated focus of the Pomodoro technique makes me more efficient at the work I am doing. When I am solely focused on one task at a time–without checking Facebook or Twitter–I work much more efficiently. Several studies have shown that multitasking causes a drop in efficiency. When I stay focused on a single task, I get much more done. I’ve written about this before, when I talked about quitting your job, but you can easily lose hours of time in a day to small distractions. Over the period of a year’s worth of time, all those wasted minutes can end up equaling weeks of lost productivity.
One huge benefit of this technique is that I am able to do most of this work without stress or guilt. Normally, when I am working, I always feel guilty about how much time I am wasting during the day. I also feel stressed about not getting as much done as I should be getting done. This situation of stress and guilt actually ends up being the perfect breeding ground for procrastination and burnout.
When I am using the Pomodoro technique, I don’t feel the guilt of wasting time, because I know that as long as I get done 10 Pomodoros in a day I have reached my productivity goal for the day. If I get more done, great.
I also don’t feel stressed about getting as much done as I should have, because how much I get done is no longer what is being measured–I’ve taken the burden off of my shoulders. My focus has shifted from results to process. I can’t control the results. Work takes as long as work takes. But, I can control the process. If I put in my 10 Pomodoros for the day and I have sufficiently prioritized my work, then I have done the best that I can do–no need for guilt, shame or stress.
Time for a break
If you are interested in getting started with the Pomodoro technique, I’d recommend checking out Pomodoro Technique Illustrated. And if you have any questions about my process and how it works, feel free to ask and I am happy to answer them in the comments below.
Also, if you liked this post and are interested in more of what I have to say about being productive and boosting your career, sign up for my weekly newsletter here. You also might want to check out the course I am putting together called “How To Market Yourself as a Software Developer.”
Far too many Agile development projects fail. It is hard to even accurately measure the number of failures because so many software development projects end up “complete” and shipped, even though:
- They took far too long to build
- The quality of what we built was poor
- What was built was not what the customer wanted
- The cost to develop was more than it should have been
Over the years, I’ve worked on many different Agile teams and have consulted for and trained many teams on Agile development. In this time, I have found 5 common problems, that if not resolved, are very likely to cause a project to be a failure.
1. Not having a product owner
Of all the things that can cause an Agile software project to fail, not having a person that is ultimately the decision maker for the product being developed is the quickest way to ensure its demise.
It doesn’t matter if you are following Scrum, doing your own Kanban style project or something else; if you want your project to succeed, you need someone who can set its direction and make decisions about the product being developed.
Think about remodeling a kitchen. If you hired a bunch of contractors to come in and do the various jobs, like: plumbing; carpentry; flooring; etc, but you don’t have someone deciding what the actual finished kitchen should look like, you are going to get a random mess.
A few skilled contractors will probably be smart enough to find the right person to ask about what should be done, but it takes more than just making arbitrary decisions about where the cabinets should go to design a kitchen. You need someone to actually come up with a real design and vision and be able to modify that vision appropriately as the project progresses and problems inevitably occur.
It seems pretty nuts to spend a huge amount of money remodeling your kitchen, but not want to invest any time or effort in either designing the finished product or hiring someone to do it.
Yet, day in and day out, I see this exact behavior in software projects. I see companies investing huge amounts of cash in Agile development, but not appointing anyone to be the true owner of what is being built and to set the vision for it.
2. Not iterating
One of the key values that Agile development brings to the software development world is iteration.
What is iteration though?
You may think it means breaking your project into 2 week sprints or iterations. While doing this can facilitate iterative development, it doesn’t mean you are actually iterating.
Confused? Let me explain.
The key to iterating is to develop a product a little bit at time. It would be accurate to describe the process of iterating on a product as evolution of a product.
Whether you believe in macroevolution or not, microevolution, or adaptation is a proven concept. The idea behind evolution is that things change gradually over time. Those small changes add up to result in a big change.
Imagine how silly it would be if evolution worked the way most “Agile” software is built.
Imagine, if you will, a fish that swims in the ocean and has some little fish babies that are born with fully functional legs. Then, those legged fish babies grow up and have fish babies that have wings.
Probably the legs and wings wouldn’t do that fish much good, nor would they be designed properly, because instead of adapting and changing over time, they just suddenly appeared.
Features should not be built in a single sprint or iteration. It is as silly as legs showing up on a fish in a single generation.
Instead, features should be evolved and grow over time. A feature shouldn’t be pushed into a single sprint or iteration and then be done. A feature should start out small and develop and evolve over time as feedback is received and customers or the product owner tries it out.
Far too many times, I see Agile development projects mistake having iterations with actually iterating the development of the product.
Don’t try to ship a completed feature at once, let it evolve over time.
3. Not breaking things down small enough
The main reason why this is so important is because it prevents procrastination. Procrastination usually occurs when either we dread some large task that will be difficult or we don’t know what to do next.
If you can break a big project up into small parts, it will seem easier to accomplish and will have clear steps of progression.
I often see software projects where the person creating the backlogs or work items is not considering the work enough before giving it to the team.
I coined a term for these kinds of backlogs: fatlogs. Fatlogs are backlogs that are not broken down small enough and often are very vague in what needs to be accomplished.
Fatlogs are notoriously difficult to estimate and waste time when trying to explain and understand them. It is critical that fatlogs be broken down into smaller actionable backlogs before being given to an Agile development team or a large amount of time will be wasted.
Many times, I have found that the creator of the fatlog could have easily broken down the work it represents into several smaller backlogs that would be easier to explain and understand, even without knowing much about software development.
I often recommend to Agile development teams that they outright reject fatlogs and send them back up the chain.
“If you can’t take enough time to clearly state what you want, it must not be that important.”
This doesn’t excuse development teams either. Development teams should break down any backlogs they get into small tasks as well.
4. Not setting done criteria
What is the first thing a waiter asks you when you order a steak?
That’s right, they ask you how you would like it done.
If the chef doesn’t know what the done criteria is for your steak, the chef will have to decide what he or she thinks the done criteria is.
You may get back a well-done steak, or a rare steak, or something in between, depending on the personal preferences of the person cooking your meat.
This isn’t a good way to cook steaks and it isn’t a good way to cook software either.
In many software projects I often encounter lots of steaks being cooked but no one defining when they are done. The backlogs most often end up being “done” when time runs out.
It is very important to have explicit unambiguous done criteria for any backlogs being worked on by an Agile development team.
This means that the product owner should be defining some level of acceptance testing. It doesn’t matter if the tests are manual tests or fully automated tests (BATs), what matters is that some criteria is defined by which the team can evaluate whether they have achieved their goal or not.
Lacking this criteria is like a parent responding with “enough” when their child asks “how many more peas do I have to eat?”
Not having done criteria is demotivating and results in the finger pointing game of why the correct thing was not built.
I have found that if you tell someone exactly what is expected of them and what criteria they will be judged by, you get much better results than just throwing them in the ring, slapping them on the back, and saying “do a good job.”
5. Not letting the team be a team
Teams are strange organisms, especially Agile teams.
There are many dynamics that affect how teams act and interact. There are many ways to foster a healthy team and many ways to create unhealthy teams.
A healthy motivated team has synergy, a unhealthy gaunt team can be less effective than all of its members are on their own.
The key to having a healthy motivated team, is letting them be mostly autonomous. Regardless of your political persuasion, you would probably agree that when a country invades another country and sets up an unelected government to oversee the people, there are going to be problems.
The same thing happens in Agile development.
I’m not saying teams shouldn’t have accountability. But if you want to run a software project in an Agile way, you have to let teams self organize and self manage to some degree.
When big brother is always watching and interfering, team dynamics become very different. Naturally, teams usually develop their own cadence, leadership and roles (this is called norms.) However, when outside forces directly interfere with how the team does things, and they don’t have the power to decide their own fate, team members begin to realize that they need to look out for themselves.
Additional agile development resources
I’ve found it to be pretty difficult to find good resources on these topics, but here are a few books I’ve found useful that address some of the issues I described above.
- Succeeding with Agile: Software Development Using Scrum – This book is focused on Scrum but it applies to many different kinds of Agile projects as well. Mike Cohn and I usually agree on most Agile subjects.
- Agile Retrospectives - I’ve found this book to be useful for getting good idea for team retrospectives that can really help to break down some barriers and help teams learn to resolve issues themselves.
- Scrumban and Kanban and Scrum – Both of these books have excellent information about combining Scrum and Kanban and have great solutions for dealing with some of the problems I described above.
What do you think? Have I missed any of the major Agile project killers? Any useful tips for dealing with them?
Let me know in the comments below.
I always hear the advice that we should tackle hard problems first.
It seems like pretty legitimate advice, and many of the reasons for saying so make sense, at least at a surface level.
The reasoning usually revolves around the idea that by tackling the hard problems first you can eliminate your biggest risk early and everything will be smooth sailing from there.
When you really think about the reasoning for solving hard problems first though, most of it is not actually reasoning, but based of off one emotion… fear.
We tend to think it is best to solve hard problems first, because we are thinking about eliminating our fear, not because we are thinking about what approach has the highest chance of success or is the most optimal.
I call this FDD, or Fear Driven Development.
And when I think about it that way, I find myself hard pressed to find a real good reason for tackling hard problems first besides being able to abort early. Which, in some cases might be good, but I’d rather focus on success.
Here are 7 reasons why it might be a good idea to tackle the hard problems last instead of first.
1. Solving easy problems first gives you momentum
When a large ball starts rolling down a hill, it picks up speed rapidly and that large ball can bust through many barriers that it couldn’t before, simply because of one thing it has after rolling down a hill that it didn’t have before—momentum.
On the converse, trying to push a heavy ball up a hill is pretty hard. And if there are barriers on the way to the top of the hill, not only do you have to fight gravity, but you have to be able to push extra hard to get through those barriers.
Life is hard enough, why make it harder?
I recently received an email from a developer that was concerned that his team wasn’t gelling and that they didn’t have the expertise in the technology needed to solve the complicated problem ahead of them.
They were going to start the project by trying to integrate this fairly complex technology and he was afraid that it would cause them a large delay before they would be able to get version 1 out.
Start with your simple problems; get working software out there as soon as possible. Not only will the team gel much more easily as they are having success and making real progress, but that momentum will help them when it is time to solve the more difficult problem.
Even if they have to throw the first version away, when they get to the hard problem, the momentum alone will make them much more likely to reach success in the end.
I could give 100 examples of how solving easy problems to gain momentum can benefit you, but you probably already know intrinsically that this is true.
Long story short, get a running start before taking a big leap.
2. Avoid solving the wrong problem
The big problem with solving the hard problems first is that the hard problems usually require a large amount of context in order to fully understand them.
It is very hard to get the right context for a hard problem when we take it out of its natural order of progression and artificially cut it to the front of the line.
You’d probably like to start a college class by taking the final exam first, so you don’t have to worry about it, but the problem with doing so is that you’ll lack the context and information to understand the questions and to know the answers.
When we try to tackle problems out of order to avoid leaving the hard problems to the end, we end up losing all of the learning and context that would help us to solve the hard problems at the end and we are much much more likely to end up solving the wrong problem, which is a complete waste of time.
3. Someone else may solve the problem for you
Sometimes procrastination is a good thing.
Sometimes, when you purposely push off solving a hard problem till the end, you end up finding that someone else already solved your problem.
I spent a few minutes trying to troubleshoot it, but nothing I was trying was working, so I had to make a decision.
I had 3 choices:
- Keep trying to solve this hard problem before moving on.
- Go on and do other videos and send off a support request to see if they could handle it.
- Make a new project and re-edit all the clips.
Choices 1 and 3 involved tackling a hard problem right then and there.
Choice 2 was to tackle easy problems and see if support could solve my hard problem for me, and if not, I would solve it at the end.
I ended up choosing option 2 and it paid off. It turned out Camtasia support was able to solve my problem. By the time I needed the project to complete my course, they had solved my hard problem for me and I didn’t waste any time upfront trying to tackle it myself.
Now it could have worked out differently; I might have had to solve the problem myself at the end, but instead of assuming I would have to and wasting perhaps a day or 2, trying to solve the problem myself, I pushed it off and kept working on easy problems and I gave someone else a chance to solve my hard problem for me.
It doesn’t happen all the time, but many times if we push off the hard problems we face, we find that by the time we get to them, someone else has already solved the problem for us.
4. Your own subconscious mind may solve the problem
When I said someone else might solve the problem for you, that someone else might actually by you—at least your subconscious mind.
Have you ever had the experience of thinking about a problem and not being able to figure it out, but then you wake up the next morning and suddenly have the answer?
It seems that our subconscious mind is more powerful than we are aware of.
In many cases, if we know of the hard problem we need to solve and have thought about it a little bit, our subconscious mind will start working on the solution, even though we are not aware.
Obviously this isn’t going to work all the time, and your subconscious mind isn’t going to write a bunch of code for you, but in many cases there is at least some benefit to throwing the problem off to our internal “worker thread.”
5. You are more committed to solving the hard problem when you risk everything you do so far
One benefit to saving the hard problem for last is that you have some extra motivation in the form of loss aversion.
We can use this knowledge to our advantage by doing the easy work first and letting our loss aversion help motivate us to solve the harder problems, because we don’t want to lose all the work we put into a project so far.
By starting with easy problem, we put some “skin in the game.”
If we try to solve the hard problems first, we have nothing to lose, so we are much more likely to give up.
6. Hard problems are often easy problems bundled together
And it turns out that many hard problems (not all) are decomposable into many small easy problems.
If you strive to never solve hard problems and to always push them off, you may actually find out that you never have to solve hard problems.
Many times we can chip away at hard problems, by taking bits of them off a little at a time and solving those easier problems. Eventually, you may find that you’ve reached the tootsie roll center of your hard problem lollipop and it is filled with chocolate candy!
Now, some problems aren’t very easily decomposable, but a good majority of problems are. Once you develop the skills to chip off bits of hard problems into smaller easy problems, the world looks like a much more conquerable place.
So, saving hard problems for last and breaking off little pieces of them as you go, can be a good strategy to help you to wear down your opponent before you have to face him.
7. Some hard problems are never truly solved
One of the big problems with tackling the hard problems first is that they tend to fill up as much time as you’ll give them.
If I give you an easy problem, like write a function to reverse a string, there isn’t much to think about. You can solve it a number of different ways and there isn’t a huge difference in the efficiency of the different methods of solving it. It is pretty unlikely you’ll spend weeks revamping your solution and thinking that it’s not quite right.
But, if I give you a problem like, create an in-memory database, not only is it a hard problem, but it has a huge number of possible solutions and can be optimized from now until the end of time. You could spend 2 weeks working on that task or 2 years.
The problem is that many hard problems don’t have a good scope to them when they are solved in isolation.
If you design an engine for a car that isn’t built yet, you won’t know when you are done.
But if you design an engine for a car and you know how much it weighs and know what kind of transmission it will use and what kind of fuel efficiency it needs to have, you can have a much clearer understanding of when your engine design is done.
If you save the hard problems for last, the scope of those hard problems will be much more defined, which will keep you from wasting valuable time over solving a problem or, like I mentioned earlier, solving the wrong problem altogether.
Baccarat is an interesting card game that you’ll find at many casinos. The objective of the game is to correctly predict whether the bank or player will win a hand.
In Baccarat the scoring for a hand is very simple, add up all the cards at their face value with face cards being worth 10 and only count the total in the ones column.
6 + 7 + J = 23 = 3
A + 4 = 5
The highest possible hand is 9 and whoever has the highest hand wins. If the player and banker have the same hand, it is a tie.
I won’t go into the details of how the number of cards are drawn is determined, but if you are interested you can find that information on Wikipedia. Basically, you end up having pretty close to a 50 / 50 chance of either the player or banker winning a hand. (Of course the house edge still is about 1.06% in the best case.)
The interesting thing about Baccarat though, is that despite the odds, despite common sense, despite the understanding that the game is completely random, people will still sit there and record every single hand and score trying to use it to look for patterns to predict future results.
These poor deluded souls actually think they are measuring something on these score cards, as if what happened in the last hand will in any way affect what will happen in the next hand.
After many years of trying to find the secret formula for measuring software development activities, I’ve come to the conclusion that trying to measure just about any aspect of software development is like trying to measure the odds of a future Baccarat hands based previous Baccarat hands.
Why we want to measure software development
It’s understandable why we want to measure software development—we want to improve. We want to find out what is wrong and fix it and we want to know when things go wrong.
After all, who hasn’t heard the famous quote:
“What gets measured gets improved.”
Don’t we all want to improve?
Somehow we get stuck with this awful feeling that the opposite is true—that what doesn’t get measured doesn’t get improved.
And of course we feel guilty about it, because we are not doing a good job of measuring our software development practices.
Just like the avid Baccarat gambler, we want to believe there is some quantifiable thing we can track, which will give us information that can give us the edge.
Sometimes the reason for wanting to measure is more
sinister practical, we want to evaluate the individuals on our team to see who is the best and who is the worst.
If we could figure out how to measure different aspects of software development, a whole world of opportunities open for us:
- We can accurately give customers estimates
- We can choose the best programming language and technology
- We can figure out exactly what kind of person to hire
- We can determine what kind of coffee produces the best code
How we try
I’ve been asked by many managers to come up with good metrics to evaluate a software development team.
I’ve tried just about everything you can think of:
- Lines of code written
- Bugs per developer
- Bugs per line of code
- Defect turn around time
- Average velocity
- Unit test code coverage percentage
- Static analysis warnings introduced
- Build break frequency
I’ve built systems and devised all kinds of clever ways to measure all of these things.
I’ve spent countless hours breaking down backlogs to the smallest level of detail so that I could accurately estimate how long it would take to develop.
I’m sure you’ve probably tried to measure certain aspects of software development, or even tried to figure out what is the best thing to measure.
It’s just too hard
No matter what I measure or how I try to measure it, I find that the actual data is just about as meaningless as notebook full of Baccarat hands.
One of the biggest issues with measuring something is that as soon as you start measuring it, it does start improving.
What I mean by this is that if I tell you that I am going to start looking at some metric, you are going to try and improve that metric. You won’t necessarily improve your overall productivity or quality, but you’ll probably find some way—intentional or not—to “game the system.”
Some managers try to get around this issue by just not telling the team what they are being measured on. But, in my opinion, this is not a good idea. Holding someone accountable to some realistically arbitrary standard without telling them what, is just not very nice at all, to put it mildly.
But really the biggest reason why it is too hard to measure aspects of software development, is that there are just way too many variables.
- Each software development project is different
- Each feature in a project is different
- Software developers and other team members are different
- From day to day even the same software developer is different. Did Jack’s wife just tell him she was cheating on him? Did Joe just become obsessed with an online game? Is Mary just sick of writing code this week?
- As you add more unit tests the build time increases
- Different team members go on PTO
- Bob and Jim become better friends and chat more instead of work
The point is everything is changing every day. Just about every aspect of software development is fluid and changing.
There is not one metric or even a set of metrics you can pick out that will accurately tell you anything useful about a software development project. (At least I have never seen one at any software development shop I’ve ever been at on consulted at.)
If you were building widgets in a factory, you could measure many qualities about that widget making process, because much of it would be the same from day to day, but with software development, you are always exploring new territory and a 1000 different variables concerning how you are developing the software changing at the same time.
Measuring without measuring
So am I basically saying that metrics in software development are completely worthless and we shouldn’t bother to track anything?
No, not exactly.
What I am saying is that trying to use metrics int the same way that we measure the average rainfall in a city, or running pace improvement by looking at its average over time, doesn’t really work in software development.
We can track the numbers, but we can’t draw any good conclusions from them.
For example, say you track defects per lines of code and that number goes up one week, what does it mean? Any number of things could have caused that to happen or it could just be a totally random fluke. You can’t really know because there isn’t a knob you can turn and say “ah, I see we turned up the coffee bitterness factor to 3 and it resulted in more bugs.” Instead there are 500 knobs and they all changed in random directions.
So, I am saying don’t look at how the numbers of any particular metric are moving from day to day or week to week and expect that it means anything at all, instead look for huge deviations, especially if they are sustained.
If all of a sudden your average team velocity dropped down to almost nothing from some very high number, you won’t know what caused it, but you’ll know that it is much more likely that there was one single knob that got cranked in some direction and you’ll at least have some idea what to look for.
You really have to treat the software development process more like a relationship than like a factory.
I don’t have a series of metrics I use to evaluate my relationship with my wife or my friends. I don’t secretly count how many times my wife sighs at me in a day and track it on a calendar to determine our relationship quality factor.
Instead what I do is talk to her and ask her how things are going, or I get a more general idea of the health of the relationship by being involved in it more.
Team retrospectives are a great way to gauge the temperature of the team. Ask the team members how things are going. They will have a pretty good idea if things are improving or slowing down and what the effectiveness level is.
Measure not, but continuously improve, yes
So kick back, don’t worry so much. I promise I won’t tell Six Sigma that you aren’t using metrics.
Instead focus on continuously improving by learning and applying what you learn. If you can’t notice enough of a difference without metrics, metrics wouldn’t have helped you anyway, because the difference would just be lost in variance anyway.
I used to be very confident in my abilities as a software developer.
I used to be able to walk up to a group of software developers and tell them exactly what they were doing wrong and exactly what was the “right” way to do things.
I used to be sure of this myself.
It wasn’t even that long ago. Heck, when I look at the blog posts I wrote 3 years ago I have to slap myself upside my head in realization of just how stupid I was.
Not only was my writing bad, but some of my thoughts seem so immature and uneducated that it feels like a completely different person wrote them.
And I wrote those posts back when I knew it all.
The more I learn, the less I know
Lately I’ve been running into situations more and more often where I don’t have a good answer for problems.
I’ve found myself much more often giving 3 pieces of advice attached with pros and cons rather than giving a single absolute—like I would have done perhaps 3 years ago.
I’ve been finding as I have been learning more and more (the past 3 years have been an accelerated pace of growth for me,) that I am becoming less and less sure of what I know and more and more convinced that there is no such thing as a set of best practices.
I’ve even spent some time postulating on whether or not commonly held beliefs of best practices would be thrown completely out the window given a significant enough motivation to succeed.
My point is that the more doors I open, the more aware I become of the multitude of doors that exist.
It is not just the realization of what I don’t know, but also the realization of weakness of the foundation I am already standing on.
Taking it out of the meta-physical
Let’s drop down out of the philosophical discussion for a bit and talk about a real example.
Perhaps the biggest quandary I struggle with is whether or not to unit test or practice TDD and its variants.
The 3 years ago know-it-all version of me would tell you emphatically “yes, it is a best practice and you should definitely do it all the time.”
The more pragmatic version of me today says, in a much more uncertain tone, “perhaps.”
I don’t want to delve into the topic in this post since I am sure I could write volumes on my ponderings in this area, but I’ve come to a conclusion that it makes sense to write unit tests for code that has few or no dependencies and that it does not make sense to do so for other code.
From that I’ve also derived that I should strive to write code that separates algorithms from coordinators.
I still even feel today that my advice is not wholly sound. I am convinced it is a better approach than 100% TDD and units tests, or no TDD and unit tests, but I am not convinced there isn’t a deeper understanding and truth that supersedes my current thoughts on the matter.
As you can imagine this is quite frustrating and unsettling.
Silver bullets and best practices
What I am coming to realize more and more is that there are no silver bullets and more surprisingly there are not even any such things as best practices.
Now I’ve heard the adage of there being no silver bullets so many times that it makes me physically sick when I hear someone say it, because it is so cliché.
But, I’ve had a hard time swallowing the no best practices pill.
I feel like when I abandon this ship then I am sailing on my life raft in the middle of a vast ocean with no sails and no sense of what direction to go.
A corner-stone of my development career has been in the learning, applying and teaching of best practices. If these things don’t exist, have I just been pedaling snake oil and drinking it myself?
Best practices are simply concrete applications of abstract principles in software development that we cannot directly grasp or see clearly enough to absolutely identify.
Breaking this down a bit, what I am saying is that best practices are not the things themselves to seek, but through the pursuit of best practices we can arrive at a better understanding of the principles that actually are unchanging and absolute.
Best practices are optimal strategies for dealing with the problems of software development based on a particular context. That context is primarily defined by:
- Language and technology choice
- Meta-game (what other software developers and perceived best practices are generally in place and how software development is viewed and understood at a given time.)
- Individual skill and growth (what keeps me straight might slow you down; depends on where you are in your journey.)
There is a gentle balance between process and pragmatism.
When you decide to make your cuts without the cutting guide, it can make you go faster, but only if you know exactly what you are doing.
Where I am now
Every time I open my mouth I feel like I am spewing a bunch of bull crap.
I don’t trust half of what I say, because I know so much of it is wrong.
Yet I have perhaps 10 times more knowledge and quite a bit more experience in regards to software development than I did just 3 years ago.
So what gives?
Overall, I think I am giving better advice based on more practical experience and knowledge, it is just that I am far more aware of my own short-comings and how stupid I am even today.
I have the curse and blessing of knowing that only half of what I am saying has any merit and the other half is utter crap.
Much of this stems from the realization that there are no absolute right ways to do things and best answers for many of the problems of software development.
I used to be under the impression that someone out there had the answer to the question of what is the right way to develop software.
I used to think that I was picking up bit of knowledge, clues, that were unraveling the mystery of software development. That someday I would have all the pieces of understanding and tell others exactly how they should be developing software.
What I found instead was that not only does nobody know the “right” way to develop software, but that it is perhaps an unknowable truth.
The best we can do is try to learn from obvious mistakes we have made before, start with a process that has had some level of success, and modify what we do based on our observations.
We can’t even accurately measure anything about software development and to think we can is just foolishness.
From story points, to velocity, to lines of code per defect and so on and so forth, all of those things are not only impossible to accurately measure, but they don’t really tell us if we are doing better or not.
So, what is my point?
My point is simple.
I have learned that not only do I not have all the answers, but I never will.
What I have learned is always subject for debate and is very rarely absolute, so I should have strong convictions, but hold onto them loosely.
And most importantly, don’t be deceived into thinking there is a right way to develop software that can be known. You can improve the way you develop software and your software development skills, but it will never be based on an absolute set of rules that come down from some magical process or technology.