Category Archives: Best Practices

Step By Step Path to Becoming a Great Software Developer

I get quite a few emails that basically say “how do I become a good / great software developer?”

These kinds of emails generally tick me off, because I feel like when you ask this kind of question, you are looking for some magical potion you can take that will suddenly make you into a super developer.

I suspect that very few people who email me asking this question really want to know how to become a great software developer, but are instead looking for a quick fix or an easy answer.

On the other hand, I think there are some genuinely sincere developers that just don’t even know how to formulate the right questions they need to ask to get to their desired future. I think those developers–especially the ones starting out–are looking for a step-by-step guide to becoming a great developer.

super hero Step By Step Path to Becoming a Great Software Developer

I thought I would make an attempt, from my experience and the best of my knowledge, to offer up that step-by-step guide.

Now, of course, I realize that there is no magical formula and that there are multiple paths to success, but I think what follows is a reasonable outline of steps someone starting out could take to reach a pretty high level of proficiency and be generally regarded as a good–perhaps even great–developer.

Step 1: Pick one language, learn the basics

Before we can run, we have to learn to walk. You walk by learning how to program in a single programming language. You don’t learn to walk by trying to learn 50 million things all at once and spreading yourself way too thin.

Too many beginning programmers try and jump into everything all at once and don’t have the patience to learn a single programming language before moving forward. They think that they have to know all the hot new technologies in order to get a programming job. While it is true that you need to know more than just the basics of a single programming language, you have to start here, so you might as well focus.

Pick a single programming language that you think you would be likely to base your career around. The programming language itself doesn’t matter all that much, since you should be thinking for the long term here. What I mean is you shouldn’t try and learn an “easy” programming language to start. Just learn whatever language you are interested in and could see yourself programming in for the next few years. You want to pick something that will have some lasting value.

Once you’ve picked the programming language you are going to try and learn, try and find some books or tutorials that isolate that programming language. What I mean by this is that you don’t want to find learning materials that will teach you too much all at once. You want to find beginner materials that focus on just the language, not a full technology stack.

As you read through the material or go through the tutorial you have picked out, make sure you actually write code. Do exercises if you can. Try out what you learned. Try to put things together and use every concept you learn about. Yes, this is a pain. Yes, it is easier to read a book cover-to-cover, but if you really want to learn you need to do.

When you are writing code, try to make sure you understand what every line of code you write does. The same goes for any code you read. If you are exposed to code, slow down and make sure you understand it. Whatever you don’t understand, look up. Take the time to do this and you will not feel lost and confused all the time.

Finally, expect to go through a book or tutorial three times before it clicks. You will not get “programming” on the first try–no one ever does. You need repeated exposure before you start to finally get it and can understand what is going on. Until then you will feel pretty lost, that is ok, it is part of the process. Just accept it and forge ahead.

Step 2: Build something small

Now that you have a basic understanding of a single programming language, it’s time to put that understanding to work and find out where your gaps are. The best way to do this is to try and build something.

Don’t get too ambitious at this point–but also don’t be too timid. Pick an idea for an application that is simple enough that you can do it with some effort, but nothing that will take months to complete. Try to confine it to just the programming language as much as possible. Don’t try to do something full stack (meaning, using all the technologies from user interfaces all the way to databases)–although you’ll probably need to utilize some kind of existing framework or APIs.

For your first real project you might want to consider copying something simple that already exists. Look for a simple application, like a To-Do list app and straight out try to copy it. Don’t let your design skills stand in the way of learning to code.

scribbles Step By Step Path to Becoming a Great Software Developer

I’d recommend creating a mobile application of some kind, since most mobile applications are small and pretty simple. Plus, learning mobile development skills is very useful as more and more companies are starting to need mobile applications. Today, you can build a mobile application in just about any language. There are many solutions that let you build an app for the different mobile OSes using a wide variety of programming languages.

You could also build a small web application, but just try to not get too deep into a complex web development stack. I generally recommend starting with a mobile app, because web development has a higher cost to entry. To develop a web application you’ll need to at least know some HTML, probably some back-end framework and JavaScript.

Regardless of what you choose to build, you are probably going to have to learn a little bit about some framework–this is good, just don’t get too bogged down into the details. For example, you can write a pretty simple Android application without having to really know a lot about all of the Android APIs and how Android works, just by following some simple tutorials. Just don’t waste too much time trying to learn everything about a framework. Learn what you need to know to get your project done. You can learn the details later.

Oh, and this is supposed to be difficult. That is how you learn. You struggle to figure out how to do something, then you find the answer. Don’t skip this step. You’ll never reach a point as a software developer where you don’t have to learn things on the spot and figure things out as you go along. This is good training for your future.

Step 3: Learn a framework

Now it’s time to actually focus on a framework. By now you should have a decent grasp of at least one programming language and have some experience working with a framework for mobile or web applications.

Pick a single framework to learn that will allow you to be productive in some environment. What kind of framework you choose to learn will be based on what kind of developer you want to become. If you want to be a web developer, you’ll want to learn a web development framework for whatever programming language you are programming in. If you want to become a mobile developer, you’ll need to learn a mobile os and the framework that goes with it.

Try to go deep with your knowledge of the framework. This will take time, but invest the time to learn whatever framework you are using well. Don’t try to learn multiple frameworks right now–it will only split your focus. Think about learning the skills you need for a very specific job that you will get that will use that framework and the programming language you are learning. You can always expand your skills later.

Step 4: Learn a database technology

Most software developers will need to know some database technology as most series applications have a back-end database. So, make sure you do not neglect investing in this area.

You will probably see the biggest benefit if you learn SQL–even if you plan on working with NoSQL database like MongoDB or Raven, learning SQL will give you a better base to work from. There are many more jobs out there that require knowledge of SQL than NoSQL.

Don’t worry so much about the flavor of SQL. The different SQL technologies are similar enough that you shouldn’t have a problem switching between them if you know the basics of one SQL technology. Just make sure you learn the basics about tables, queries, and other common database operations.

I’d recommend getting a good book on the SQL technology of your choice and creating a few small sample projects, so you can practice what you are learning–always practice what you are learning.

You have sufficient knowledge of SQL when you can:

  • Create tables
  • Perform basics queries
  • Join tables together to get data
  • Understand the basics of how indexes work
  • Insert, update and delete data

In addition, you will want to learn some kind of object relational mapping technology (ORM). Which one you learn will depend on what technology stack you are working with. Look for ORM technologies that fit the framework you have learned. There might be a few options, so you best bet is to try to pick the most popular one.

Step 5: Get a job supporting an existing system

Ok, now you have enough skills and knowledge to get a basic job as a software developer. If you could show me that you understand the basics of a programming language, can work with a framework, understand databases and have built your own application, I would certainly want to hire you–as would many employers.

maintainance Step By Step Path to Becoming a Great Software Developer

The key here is not too aim to high and to be very specific. Don’t try and get your dream job right now–you aren’t qualified. Instead, try and get a job maintaining an existing code base that is built using the programming language and framework you have been learning.

You might not be able to find an exact match, but the more specific you can be the better. Try to apply for jobs that are exactly matched to the technologies you have been learning.

Even without much experience, if you match the skill-set exactly and you are willing to be a maintenance programmer, you should be able to find a job.

Yes, this kind of job might be a bit boring. It’s not nearly as exciting as creating something new, but the purpose of this job is not to have fun or to make money, it is to learn and gain experience.

Working on an existing application, with a team of developers, will help you to expand your skills and see how large software systems are structured. You might be fixing bugs and adding small features, but you’ll be learning and putting your skills into action.

Pour your heart into this job. Learn everything you can. Do the best work possible. Don’t think about money, raises and playing political games–all that comes later–for now, just focus on getting as much meaningful productive work done as possible and expanding your skills.

Step 6: Learn structural best practices

Now it’s time to start becoming better at writing code. Don’t worry too much about design at this point. You need to learn how to write good clean code that is easy to understand and maintain. In order to do this, you’ll need to read a lot and see many examples of good code.

Beef up your library with the following books:

Language specific structural books like:

At this point you really want to focus your learning on the structural process of writing good code and working with existing systems. You should strive to be able to easily implement an algorithm in your programming language of choice and to do it in a way that is easy to read and understand.

Step 7: Learn a second language

At this point you will likely grow the most by learning a second programming language really well. You will no doubt, at this point, have been exposed to more than one programming language, but now you will want to focus on a new language–ideally one that is significantly different than the one you know.

This might seem like an odd thing to do, but let me explain why this is so important. When you know only one programming language very well, it is difficult to understand what concepts in software development are unique to your programming language and which ones transcend a particular language or technology. If you spend time in a new language and programming environment, you’ll begin to see things in a new way. You’ll start to learn practicality rather than convention.

As a new programmer, you are very likely to do things in a particular way without knowing why you are doing them that way. Once you have a second language and technology stack under your belt, you’ll start to see more of the why. Trust me, you’ll grow if you do this. Especially if you pick a language you hate.

Make sure you build something with this new language. Doesn’t have to be anything large, but something of enough complexity to force you to scratch your head and perhaps bang it against the wall–gently.

Step 8: Build something substantial

Alright, now comes the true test to prove your software development abilities. Can you actually build something substantial on your own?

If you are going to move on and have the confidence to get a job building, and perhaps even designing something for an employer, you need to know you can do it. There is no better way to know it than to do it.

Pick a project that will use the full stack of your skills. Make sure you incorporate database, framework and everything else you need to build a complete application. This project should be something that will take you more than a week and require some serious thinking and design. Try to make it something you can charge money for so that you take it seriously and have some incentive to keep working on it.

Just make sure you don’t drag it out. You still don’t want to get too ambitious here. Pick a project that will challenge you, but not one that will never be completed. This is a major turning point in your career. If you have the follow-through to finish this project, you’ll go far, if you don’t… well, I can’t make any promises.

Step 9: Get a job creating a new system

Ok, now it’s time to go job hunting again. By this point, you should have pretty much maxed out the benefit you can get from your current job–especially if it still involves only doing maintenance.

new Step By Step Path to Becoming a Great Software Developer

It’s time to look for a job that will challenge you–but not too much. You still have a lot to learn, so you don’t want to get in too far over your head. Ideally, you want to find a job where you’ll get the opportunity to work on a team building something new.

You might not be the architect of the application, but being involved in the creation of an application will help you expand your skills and challenge you in different ways than just maintaining an existing code base.

You should already have some confidence with creating a new system, since you’ll have just finished creating a substantial system yourself, so you can walk into interviews without being too nervous and with the belief you can do the job. This confidence will make it much more likely that you’ll get whatever job you apply for.

Make sure you make your job search focused again. Highlight a specific set of skills that you have acquired. Don’t try to impress everyone with a long list of irrelevant skills. Focus on the most important skills and look for jobs that exactly match them–or at least match them as closely as possible.

Step 10: Learn design best practices

Now it’s time to go from junior developer to senior developer. Junior developers maintain systems, senior developers build and design them. (This is a generalization, obviously. Some senior developers maintain systems.)

You should be ready to build systems by now, but now you need to learn how to design them.

You should focus your studies on design best practices and some advanced topics like:

  • Design patterns
  • Inversion of Control (IOC)
  • Test Driven Development (TDD)
  • Behavior Driven Development (BDD)
  • Software development methodologies like: Agile, SCRUM, etc
  • Message buses and integration patterns

This list could go on for quite some time–you’ll never be done learning and improving your skills in these areas. Just make sure you start with the most important things first–which will be highly dependent on what interests you and where you would like to take your career.

Your goal here is to be able to not just build a system that someone else has designed, but to form your own opinions about how software should be designed and what kinds of architectures make sense for what kinds of problems.

Step 11: Keep going

At this point you’ve made it–well, you’ll never really “make it,” but you should be a pretty good software developer–maybe even “great.” Just don’t let it go to your head, you’ll always have something to learn.

How long did it take you to get here? I have no idea. It was probably at least a few years, but it might have been 10 or more–it just depends on how dedicated you were and what opportunities presented themselves to you.

A good shortcut is to try and always surround yourself with developers better than you are.

What to do along the way

There are a few things that you should be doing along the way as you are following this 10 step process. It would be difficult to list them in every step, so I’ll list them all briefly here:

Teach - The whole time you are learning things, you should be teaching them as well. It doesn’t matter if you are a beginner or expert, you have something valuable to teach and besides, teaching is one of the best ways to learn. Document your process and journey, help others along the way.

Market yourself - I think this is so important that I built an entire course around the idea. Learn how to market yourself and continually do it throughout your career. Figure out how to create a personal brand for yourself and build a reputation for yourself in the industry and you’ll never be at want for a job. You’ll get decide your own future if you learn to market yourself. It takes some work, but it is well worth it. You are reading this post from my effort to do it.

Read - Never stop learning. Never stop reading. Always be working your way through a book. Always be improving yourself. Your learning journey is never done. You can’t ever know it all. If you constantly learn during your career, you’ll constantly surpass your peers.

Do - Every stop along the way, don’t just learn, but do. Put everything you are learning into action. Set aside time to practice your skills and to write code and build things. You can read all the books on golfing that you want, but you’ll never be Tiger Woods if you don’t swing a golf club.

Want more?

Sign up here to become a Simple Programmer. It’s free and over 4,500 other developers have shown their support for making the complex simple.

 

 

 

Test Automation Framework Architecture

Test automation framework architecture efforts are often complete failures.

It’s true. I’ve worked with many companies who have given up on creating a good test automation framework architecture, because after investing a large amount of time and money and resources in doing it the wrong way, they have incorrectly assumed the entire effort is not cost effective.

In this post, I’m going to simplify the process of creating a test automation framework architecture and show you that—if you do it right—it can be a very good investment.

Test automation framework architecture basics

Let’s start off by talking about what the goals of a successful test automation framework architecture should be.

There are two goals I am interested in when creating a test automation framework:

  1. Having the ability to easily create simple automated tests that use the framework.
  2. Decoupling cosmetic application changes from the tests so that when the application changes, the tests do not all have to be updated

A good test automation framework architecture should at the very least provide these two important services. We create a test automation framework to allow us to support the creation of tests and to keep those tests from being dependent on the actual UI of our application—as much as possible.

I’m going to break down these goals just a little more to make sure we are all on the same page.

test automation framework thumb Test Automation Framework Architecture

Creating simple automated tests

First, we’ll start with having the ability to easily create simple automated tests that use the framework. Why do we care about this goal? And why would this be the responsibility of the framework?

I have found that one of the biggest failure points of test automation efforts is test complexity. The more complex the tests are, the harder they are to maintain. If tests are difficult to maintain, guess what? They won’t be maintained.

So, we really want to make sure that when we create a test automation framework architecture, we focus on making sure that test automation framework makes it as easy as possible for someone to create tests using it.

It is always a better choice to put the complexity into the framework instead of into the tests. The framework is usually maintained by developers and does not contain repeated code. But, tests are often created by QA or even business folks and the code found in tests is usually repeated across many tests, so it is vital to reduce the complexity on the tests, even if it means a large increase of complexity in the framework itself.

When I create a test automation framework architecture, my goal is to make it so the tests can read as close as possible to plain English. This requires some effort to pull off, but it is well worth it.

Decoupling the UI from the tests

Next, let’s talk about what I mean by decoupling cosmetic application changes from the tests.

This is also a vital goal of any test automation framework architecture, because if you fail to decouple the UI of an application from the tests, every single UI change in an application will cause possibly hundreds or thousands of tests to have to be changed as well.

What we really want to do is to make it so that our test automation framework architecture creates an abstraction layer that insulates the tests from having to know about the actual UI of the application.

At first this seems a little strange, especially when I tell automation engineers or developers creating an automated testing framework not to put any Selenium code into their actual tests.

All of the Selenium examples show you creating tests that directly use a web browser driver like Selenium, but you don’t want to write your tests that way—trust me on this one.

(By the way, I haven’t found a good book on creating an actual test automation framework architecture—I’ll probably write one—but, for now if you are looking for a good book on Selenium that starts to go into the topics I discuss here, check out Selenium Testing Tools Cookbook.)

Instead, what you want to do is to make it so the test automation framework is the only code that directly interacts with the UI of the application. The tests use the framework for everything they want to do.

So, for example:

Suppose you are creating a simple test to check to see if a user can log into your application.

You could write a test that looks something like this: (C# and Selenium in this example)

var loginInput = driver.FindElement(By.Id("txtUsername"));
loginInput.SendKeys("Joe");

var passwordInput = driver.FindElement(By.Id("txtPassword"));
passwordInput.SendKeys("$ecret");

var loginButton = driver.FindElement(By.Id("btnLogin"));
loginButton.Click();

But, what is going to happen when you change the ID of your “User Name” field? Every single test that uses that field will break.

On the other hand, if you properly create a test automation framework architecture that abstracts the UI away from the tests themselves, you would end up with a much simpler and less fragile test, like this:

LoginPage.GoTo();
LoginPage.LoginAs("Joe").WithPassword("$ecret").Login();

Now, if the ID of your “User Name” field changes, you’ll just change the code in your automated testing framework in one place, instead of changing 100 tests that all depended on the specific implementation of the UI.

A simple test automation framework architecture

Once you understand those two very important goals and why they are important, it is easier to think about how you should design a test automation framework architecture.

I’ve created quite a few test automation framework architectures, so I’ll give you a basic design I use for most of them.

Take a look at this diagram:

test automation framework architecture thumb Test Automation Framework Architecture

Here you can see that there are four layers to my test automation framework architecture.

Frist, we have the browser layer or the web app itself. This just represents your actual application.

Next, we have the Selenium or web driver layer. This layer represents your browser automation tool. Selenium is basically just a framework for automating a browser. It doesn’t know anything about testing, it is an API that lets you interact with the browser programmatically. (By the way, you don’t have to use Selenium. I just use it as an example here, because it is the most popular browser automation framework.)

After that, we have the framework layer. This is the actual framework you create which uses Selenium, or whatever web driver you want, to actually automate your application. The framework is acting as an abstraction between your application and the tests. The framework knows about the UI of your application and how to interact with it using Selenium. It is your job to create this layer.

Finally, we have the actual tests. The tests use the framework to manipulate your application and check the state of the application. These tests should be simple to understand and should not have to know anything about the actual implementation of the UI of your application. These tests should rely on the framework to give them the ability to do whatever they need to do in your application.

Now what?

Obviously, creating a test automation framework architecture is more complicated than what I have shown you in this article, but with these basics you should be off to a good start.

If you want a more in-depth and detailed explanation as well as an example of how exactly to create a full test automation framework architecture, you might want to check out my Pluralsight video on the topic:

Creating an Automated Testing Framework With Selenium

I also frequently blog about test automation topics and other topics related to becoming a better and more profitable software developer.

Sign up here to make sure that you don’t miss any of my posts and get access to the content I only share with my email list. (Don’t worry I hate spam as much as you do.)

If you have any question, post them in the comments below.

Not Everything Is 80-20, Don’t Blindly Follow Pareto’s Law

There is a useful observation about the world that is often applied to software development called the Pareto principle or Pareto’s law.

This principle suggests that in many situations 80% of the results come from 20% of the causes.

For example, Pareto had realized that 80% of the land in Italy, during his time, was owned by 20% of the population.

111613 0015 NotEverythi1 Not Everything Is 80 20, Don’t Blindly Follow Pareto’s Law

Since then, many people, including Pareto himself, have applied the same observation of this 80-20 rule to many other areas of life from economics, to business and even software development.

But, I’ve found that we rely on Pareto’s law just a little bit too much.

The problem with generalizations

The biggest problem I have with Pareto’s law is that is applied way too often to too many situations. In many cases, especially in software development, Pareto’s law becomes a self-fulfilling prophecy—the more you look for Pareto’s law, the more you magically seem to find it.

None of this is to say that Pareto’s law isn’t a real thing—of course it is. If you go and take a look at real hard numbers about distributions of things, you’ll find Pareto’s law all over the place in your statistical data. But, at the same time, if you go and look for the number 13, you’ll find an alarming number of occurrences of that two digit number above all others as well.

It is very tempting to force things that don’t quite fit generalizations into those generalizations. How often do we use the phrases “always” and “never?” How often do we fudge the data just a little bit so that it fits into that nice 80-20 distribution? 82% is close enough to 80 right? And of course 17.5% is close enough to just call it 20 after all.

Not only can you take just about any piece of data and make it fit into Pareto’s law by changing what you are measuring a little bit and fudging the numbers just a little if they are close enough, but you can also take just about any problem domain and easily, unconsciously, find the data points which will fit nicely into Pareto’s law. There is a good chance you are doing this—we all are. I do it myself all the time, but most of the time I am not aware of it.

I’ve found myself spouting off generalizations about data involving Pareto’s law without really having enough evidence to back up what I am saying. It is really easy to assume that some data will fit into Pareto’s law, because deep down inside I know I can make it fit if I have to.

Seeing the world through Pareto colored glasses

111613 0015 NotEverythi2 Not Everything Is 80 20, Don’t Blindly Follow Pareto’s LawYou might think there is no harm in fudging numbers a bit and finding more and more places to apply Pareto’s law. But, looking at the world and blindly assuming all data falls into a distribution of 20 percent of the causes being responsible for 80 percent of the effects, is like walking around with blinders on your eyes—you are only seeing part of reality and even the reality you are seeing tends to be a bit distorted.

Again, this doesn’t mean that Pareto’s law isn’t correct a large amount of the time, but it means that when you are just assuming that any data that appears to obey this law will, or worse yet, that all data MUST obey this law, you are severely limiting your perspective and restricting your options to those that already fit your preconceived ideals.

Sometimes I wish I had never heard of Pareto’s law, so that I wouldn’t be subject to this bias.

Let me give you a bit of a more concrete example.

Suppose you blindly assume that 80% of your application’s performance bottleneck comes from 20% of your code. In that case, you might be right, but you might also be wrong. It is entirely feasible that there are some parts of your code that contribute more or less to the performance of the application. It is also pretty likely that there are some bottlenecks or portions of code that heavily impact the performance of your application. But, if you go in with the assumption that the ratio is 80-20, you may spend an inordinate amount of time looking for a magical 20% that doesn’t exist instead of applying a more practical method of looking for what the actual performance problems are and then fixing them in order of impact.

The same applies for bugs or testing. If we blindly assume that 20% of the code generates 80% of the bugs, or that 20% of our tests test 80% of our system, we are making pretty large conclusions about how our software works that may or may not be correct. What happens when you fix all the bugs caused by the 20% of code that generates 80% of them? Does a new section of code now magically produce 80% of the bugs? If 20% of your test cases test 80% of your code, can’t you just create those ones? Why create another 80% to only test another 20%? And if you did follow that advice, then wouldn’t you have the situation where 100% of your tests tested 80% of your code?

The problem is when you start applying and assuming that Pareto’s law applies blindly, you start making all kinds of incorrect assumptions about data and only see what you expect.

So, was Pareto wrong?

In short, no. He wasn’t wrong. Pareto’s principle is a thing. In general, in many cases, it is useful to observe that a small amount of causes are responsible for a majority of effects.

But, it is not useful to apply this pattern everywhere you can. The observation of the data should guide the conclusion and not the other way around.

I find it more useful, especially in software development, to ask the question “is it possible to find a small thing that will have a great effect?”

A good book on this very subject is The 4 Hour Chef. Although I don’t always agree with Tim Ferris, he is definitely the master of doing more with less and talks frequently about concepts like minimum effective dosages.

In other words, given a particular situation, can I find a small thing I can do, or change I can make, that will give me the biggest bang for my buck?

Sometimes the answer is actually “no.” Sometimes, no matter how hard we try, we just can’t find a minority that influences the majority. Sometimes the bugs are truly evenly distributed throughout the system. Sometimes the contributions of team members are fairly equal. One team member is not always responsible for 80% of the results.

And let’s not forget about synergy. Which basically is when 1 + 1 is equal to 3 or more. Sometimes the combination of things together makes the whole and separating out the parts at all greatly reduces the function.

For example: eggs, sugar, flour and butter can be used to make cake, and you could say that 80% of the “cakiness” comes from 20% of the ingredients, but if you leave one of those ingredients out, you’ll quickly find that 100% of them are necessary and it doesn’t even make sense to try and figure out which ingredient is most important, because alone each ingredient functions much differently than they do all together.

In software development this is especially true. Often in complex systems all kinds of interactions between different components of a system combine together to create a particular performance profile or bug distribution. Hunting for the magical 20% in many cases is as futile as saying eggs are responsible for 80% of the “cakiness” of cake.

Leaky Abstractions Are Holding Us Back

Let’s just get right into it, shall we?

What is an abstraction?

Before we can talk about leaky abstractions, and why they are bad, let’s define what exactly an abstraction is.

An abstraction is when we take a complicated thing and we make it simpler by generalizing it based on the way we are using or talking about that thing.

We do this all the time. Our brains work by creating abstractions so that we can deal with vast amounts of information.

abstraction thumb Leaky Abstractions Are Holding Us Back

We don’t say “my Dell Inspiron 2676 with 2 GB of RAM and a 512 GB hard drive, serial number 12398ASD.” Instead we say simply “my computer.”

In programming, abstractions let us think at higher levels. We create abstractions when we give a bunch of lines of code that get the username and password out of a textbox and log the user into our system, a name; login. And we use abstractions when we utilize APIs or write to the “file system.”

Without abstractions we’d have a really hard time managing the complexity of any software system. Without abstractions, it would be pretty difficult to write any kind of non-trivial application.

If you want to feel what it is like to program with fewer abstractions, try serving up a web page with assembly code. If you want to feel what it is like to program with almost no abstractions, do the same exercise, but write your code in machine code… 0s and 1s.

How do abstractions “leak?”

Abstractions leak when you have to understand the lower level concept to use the higher level concept.leaking thumb Leaky Abstractions Are Holding Us Back

This means the details of the lower level concept that are being abstracted away are leaking up through the higher level concept.

Ever get a cryptic error message when an application you are using crashes? That is a leaky abstraction.

“Memory location 00FFE133 could not be written to” is just bad news. It means that in order to understand what is going wrong with your application, you now have to understand the inner workings of the application. Your abstraction has effectively “leaked.”

Abstractions leak whenever you are forced to pull back the curtains and see the gnomes running in hamster wheels spinning the cogs.

Why “leaking” is bad

I’m often surprised when I hear someone say “I like how you didn’t hide the details of what is really going on” about some code or some technology.

Not being able to hide the details of what is really going on points to an inability to create the proper abstraction, which in my opinion, should be water tight.

It is not a good thing that we can see what is really going on. No, it is actually a very bad thing, because it forces us to think at the higher level some of the time, but have to switch gears to drop down to the lower level some of the time.

Leaky abstractions prevent us from being able to comfortably drive the kids to school; we have to constantly keep looking in our rear view mirror and checking to see if those rowdy kids who insist on sitting at the back of the bus are causing trouble again.

A good abstraction hides the details so well that we don’t ever have to think about them. A good abstraction is like a sturdy step on a ladder, it allows us to build another step above us and climb up it. A leaky abstraction is like a wobbly rotted wood rung; put too much weight on it and you’ll fall through—it traps you at the current step—you can’t go any higher.

Shouldn’t we understand how things really work?

No! Stop already! Enough with the guilt trips and unnecessary burdens. You are defeating the purpose of abstractions and stunting further growth.

You only “need” to know the details if you want or need to know the details.

I know that many people want to know how things really work, myself included for many things, but in many cases I don’t need to understand how things really work.

For example, I can buy and use a watch without understanding how the watch works. It really is better that way. I don’t need those details floating around in my head. I get no benefit from knowing what is going on inside my watch and frankly, I don’t want to know.

Personally, I have a similar view on cars, although I know that many other people don’t, so that is why I didn’t use it as my first example.car engine thumb Leaky Abstractions Are Holding Us Back

When I buy a car, I just want to drive it. I don’t want to remember when to rotate tires, check fluids, etc. Waste of my time. I don’t want to know how the engine works or anything about it besides I have to put gas in, turn a key, and press a few pedals to make the thing go. I know I could save money servicing my own car, but you know what? I’d rather pay money than have to do it myself. And since that is the case, as long as I can find someone I trust to work on my car, I don’t want that abstraction to leak.

The same with machine code and compilers. Yes, it is interesting to understand how memory mapping and CPUs and logic gates work and how my C# code gets compiled into MSIL which is interpreted by the CLR and so forth and so on, until it ends up as electrons operating on registers in my CPU. And I learned about how much of that works, because I was interested, but let’s be completely honest here. That abstraction is really good. It is so good and so air-tight, that I don’t need to know all of that crap in order to write a web app. If I did, I’d probably find another profession, because my brain would explode every time I tried to write the simplest piece of code.

By the way, if you do want to understand what is happening behind many of those abstractions, I highly recommend Charles Petzold’s book: “Code: The Hidden Language of Computer Hardware and Software”.  (This is the kind of book you read for “fun,” not because you need to know this information.)

Leaking abstractions in technology

Too many technologies and frameworks today are leaky, and this is bad– really bad.

Want an example? Whatever happened to ORM (Object Relational Mapping?)

Nothing. Nothing happened. It was invented, oh, like thousands of years ago and while lots of other technologies and frameworks grew up and got jobs and left the nest, ORM is still a fat unemployed balding 30 year old man playing WOW all day in his mum and dad’s basement.

Don’t get me wrong. I use ORMS; I sometimes like to think I like ORMS. But, I can’t use them well because I have to understand SQL and what is going on in my relational databases anyway. What good is that?!

Seriously. This is a completely failed abstraction. The leakiness of it makes it actually more work to learn an ORM framework, because in learning one you must also learn hairy details of SQL and understand what N+1 means and so on. You can’t just fire up your ORM and put in the Konami code, you have to blow the cartridge out and fiddle with the reset button half a dozen times. Useless.

The other major example that is holding us back today is MV* in JavaScript. Look, I’m not trying to bash JavaScript this time. I’m actually starting to like JavaScript, but it is total bullcrap that in order to write a single page web application today, you have to understand 500 other different technologies and how JavaScript is working under the covers. If we are going to do MV* in JavaScript– and I am actually starting to become a believer in this truth—then we gotta seal up the abstraction and put what we need into the language or the DOM or the technology stack itself. Bolting things on to create abstractions that are ultimately leaky works for now, but it is like that rotten creaky step in the ladder, eventually we are going to fall through it and we aren’t going to be able to climb any higher until we replace that step.

I used to be convinced replacing that step meant getting rid of JavaScript and using something else superior that does what we want, but maybe it just means making JavaScript and browser APIs themselves evolve. I don’t care how we do it, but we aren’t going to advance until we do.

I could go on and on with examples of leaky abstractions in technology and how they are holding us back, but I think you get the point by now.

Leaky abstractions are bad. They don’t do anything good for us besides require us to know more and add more points where things can go wrong. We’ve got to seal up these abstractions, if we want to be able to keep climbing the software development ladder. You can’t keep building a tower if you have to worry about every single block beneath you. You eventually get to a point where you can’t keep it all straight and your tower comes crashing to the floor.

Leaky abstractions and you

So, what can you do about it?

Not much as far as ORMs and JavaScript. But, next time you write some code, seriously think about whether or not you are creating a leaky abstraction.

You really have to make a choice. Am I going to hide the details so that no one needs to know about them, or am I going to expose the whole thing? Either make a really good abstraction that doesn’t require someone using your code to understand what is going on underneath that abstraction or don’t attempt to make an abstraction at all.

It takes more effort to create good abstractions. Sometimes the effort required is orders of magnitude greater. But, if other people are going to use your abstraction, to save the mental burden of understanding what is happening beneath it is going to go a long way in making up for your extra effort.

10 Steps to learn anything quickly course

By the way, I just released by 100% 10 Steps to Learn Anything course.  If you want free access to it, just signup to my newsletter here.  It is a totally free video course and optional workbook that reveals the process I use to quickly learn technologies.

Creating an Automated Testing Framework With Selenium

I have another new course on Pluralsight, check out:

Creating an Automated Testing Framework With Selenium

2013 09 28 16 31 58 Creating an Automated Testing Framework With Selenium

I am very excited to finally get this course out.  Many viewers have been asking me to make a comprehensive course that actually shows you how to build a real automation framework, and I finally did it in this course.

I reveal all my tricks and tips here and tell you exactly what I have done in the past to build successful automation frameworks.  It was a lot of fun to create this course and it brought me back to the glory days of creating automated tests.  (This is some serious fun!)

Anyway, check out the course and let me know what you think.

Here is the official course description:

Learning how to use a tool like Selenium to create automated tests is not enough to be successful with an automation effort. You also need to know how to build an automation framework that can support creating tests that are not so fragile that they constantly break. This is the real key to success in any automation effort.

In this course, I will reveal every secret I know from creating several successful automation frameworks and consulting on the creation of others. I will show you exactly, step-by-step how to create your own automation framework and I will explain to you the reasoning behind everything we are doing, so you can apply what you learn to your own framework.

We’ll start off this course by going over the basics of automation and talking about why it is so important as well as discuss some of the common reasons for success and failure.

Then, I’ll take you into the architecture of an automation framework and show you why you need to pay careful attention to the structure of any framework you build and give you some of the underlying design principles I use when creating an automation framework.

After that we’ll be ready to start creating a framework. In the next few modules, I’ll show you how to create a real automation framework capable of automating the WordPress blogging platform administrative console. We’ll start off by creating smoke tests and using those smoke tests to build out our initial framework.

Then, we’ll expand the capabilities of our framework as we create more tests and learn how to use techniques like dummy data generators to make our tests as simple and easy to read as possible.

Finally, I’ll take you through some best practices and tips that cover topics like scaling out, working in Agile environments and other important issues you are likely to face. If you are responsible for an automation project for a web application or you want to start using automation, you’ll definitely want to check this course out.

Why I Don’t Do Code Katas

Update: be sure to check out this comment chain to see a bit more clarification on what code katas really are.  Thanks for Cory Foy for pointing out that I was unfairly calling all code katas as solving the same programming problem over and over again.  Only some people treat code katas that way and this post is about why that kind of code kata is not effective.  I also updated a few words in this post to reflect this point.

I don’t want to seem like I am bragging, but there is something I just have to get off my chest.

But, before I tell you what amazing skill I have mastered through countless hours of boring practice, let me tell you a bit about my training schedule.

Everyday I get up in the morning. I put on my shoes. I begin my practice.
Usually I start with a small warm-up, just to get the blood flowing on the way to my office at the end of the hall.

Later in the day, I’ll wander out of my office for lunch and practice more on the way to the kitchen.

Right now as I am typing out this blog post, I am on the treadmill honing my craft one painstaking repetitive step at a time.

That is right, you may have guessed it by now—I am a master walker.

All my life I have been practicing this humble skill.  Each and every day I practice my secret craft.  I see other fools riding around on mopeds and motorized scooters sheepishly talking of their perceived skill at bipedal transportation.  But, although outwardly I acknowledge their words, inwardly I know that I am special, because I hone my skill to a craft.

Sounds ludicrous?  That is what it sounds like to me when some people talk about doing code katas.

Code Katas

What is a Code Kata you may ask?  Some people think a Code Kata is when you solve the same programming problem over and over again, thinking you are actually practicing something other than using shortcuts in your IDE. (To be fair, not all people view Code Katas that way.)code kata thumb Why I Dont Do Code Katas

These kinds of Code Katas come from a failed attempt to forcibly simulate the same kind of practicing that a musician or athlete would perform in order to become better at their vocation.

Why Code Katas aren’t effective

Don’t get me wrong, the intention behind Code Katas seems to be well placed, and I am all for software craftsmanship (although not the elitism that sometimes accompanies it.)  But, just as walking every single day doesn’t make you a master walker, and driving a car every day doesn’t make you a superior driver, solving the same sets of programming problems over and over again won’t make you a master programmer.

If you still have your doubts, consider old people.  Old people are not master walkers or master drivers, yet they have been doing both of those activities day in and day out for many years.  Not only do they not master these skills, the skills actually atrophy—they become worse at them, and not just because their muscles and reflexes become worse from age, it is because repeating the same thing over and over again without an increasing level of challenge actually results in the mind becoming complacent with the activity which leads to the eventual degradation of it.  (At least, this appears to be the case to me.)

Don’t lose hope though, a little later on I will tell you a real way to practice your craft in software development—one that will give you results—but, first let’s dissect the cases of walking and driving a little bit further to see why repeated practice of those activities seems to have diminishing returns.

Doing the same thing over and over again doesn’t make us better at it

We first learn from walking and driving, because everything is new and challenging.  Although walking seems like a repetitive process, it is actually the lack of repetition that provides the challenge our brain and body need to actually benefit from it.

As a toddler learning to walk or a teenager learning to drive, we are constantly bombarded with new scenarios and challenges that we haven’t faced before.  What happens when the dog runs by and brushes our leg?  How does this affect our balance and what do we have to do to adjust?  What about this new surface?  The carpet seems to absorb our feet and create more friction when we attempt to walk on it.  How do we compensate to balance on this surface?  Walking with socks on the floor gives us less grip, etc.learning to walk thumb Why I Dont Do Code Katas

But, as an adult, or experienced walker or driver, we are not challenged in nearly the same way;  we’ve already seen most of the scenarios.  Our daily commute to work rarely presents us with an unfamiliar stimulus or problem to solve.  We operate on autopilot, automatically walking and driving to our destinations without much effort or thought.

Repeatedly writing the code to solve the same programming problem over and over again is exactly the same thing, except unless you are introducing new challenges by adding constraints and twists, you are only benefiting from the exercise the first, and perhaps, the second time, you undertake the activity.

Now, not all programmers who do Code Katas repeat the exact same code over and over again.  Some of them attempt to solve the same problem, but this time not using any loops or limited the line count to a certain number or some other similar constraint.   This kind of practice is actual worthwhile practice which challenges the mind and introduces new scenarios that create new neural pathways in the brain and strengthen others. (Here is a great example of book that encourages you to do just that.  I highly recommend working through the problems in this book.)

Simply typing the same code into the same IDE over and over again for 10 minutes a day may make you feel relaxed and calm or even like a concert violinist arduously practicing their scales, but at the very best it will make you faster at using your IDE and typing code; skills which can be easily practiced through the course of a normal day’s work.  If you want to be really good at writing the code to sort lists of words and calculate prices for grocery store items in a cart, go ahead, be my guest.

How real mastery is achieved

If you are resisting what I am saying at this point, it is likely because you still mistakenly believe that walking and driving are in a different category than playing music, sports and programming.  But, the truth of the matter is that walking, running, music, sports, and programming are in the same category, or rather, there are no categories.

It turns out there are actually master walker and master drivers.  One sport that is not all that widely known goes by the name of Parkour or Freerunning.  Don’t believe me, check out this video for an example.  In this sport, the athlete becomes exceptionally good at traversing around everyday locations on their own two feet.  It is amazing what some of these practitioners can do—just watch the video if you don’t believe me.

And as for driving, I probably don’t need to convince you that Nascar is “kind of a big deal.”

My point is that you are not going to become a Parkour expert or Nascar driver by just walking around your neighborhood or driving your car to and from work, even if you do tend to drive like a bat out of hell.  To get to that level of skill in those seemingly mundane activities, you have to constantly practice at a higher level of difficulty and continually introduce new challenges.

In fact, when we actually look at what real musicians and sports athletes do, it is much of the same.  I seriously doubt many musicians you hear on the radio or see in concert repeatedly played “bah bah black sheep” or “twinkle twinkle little star” on their instruments until one day they achieved mastery.  And in the professional sports world, achievement only comes through repeatedly pushing beyond one’s limits day after day.

The point is this: if you want to get better at something, repeating practice alone is not enough.  You must practice with increased difficulty and challenge.

How to improve your programming skills

So, how then do you become a master craftsman at the vocation of software development?  If you don’t use Code Katas, how do you practice your craft?
Let me start off by giving you the best example I have ever seen.

This is Jennifer Dewalt’s blog; she decided to learn to code by building 180 websites in 180 days.  While you are sitting at your keyboard typing the same 20 lines of code into your IDE over and over again, Jennifer is creating a new thing every single day and facing a brand new challenge.

Who do you think will improve their skills more after 180 days, you or Jennifer?  I know who I’d hire without even thinking twice.

I get lots of emails asking me about the best way to learn programming or how someone can improve their skills.  Recently, I’ve gotten a lot of emails asking about how to learn Android development.  Do you know what I tell all these inquisitive minds?

Make Android apps.

But, umm, what book can I read?

No book, just come up with an idea and try to create it.  Figure out what you need to do along the way.  When you get stuck, go look for the answer or seek help.
But, I need to know what I am doing first.

Says who?  Do and then learn.  Learn and do at the same time, reading a book or repeating the same exercise over and over again won’t make you smarter or more skillful; constantly challenging your mind and current skills will.

Want to learn more?

I’ve got plenty more to say on the topic and many more techniques that are way more effective then Code Katas which you can use to hone your programming skills.  I can’t fit it into a single post like this, but I’ll be putting it into the top secret project I am working on to help developers boost their careers and learn to market themselves.

If you’d like to be one of the first to be notified when it is available, sign up here.  I’ll be including lots of practical valuable information that you probably won’t find anywhere else.

So, what do you think?  Do you do Code Katas right now?  If so, do you think they are actually useful?  Let me know in the comments below.

5 Things That Will Make Your Agile Development Project FAIL

Far too many Agile development projects fail. It is hard to even accurately measure the number of failures because so many software development projects end up “complete” and shipped, even though:

  • They took far too long to build
  • The quality of what we built was poor
  • What was built was not what the customer wanted
  • The cost to develop was more than it should have been

Over the years, I’ve worked on many different Agile teams and have consulted for and trained many teams on Agile development. In this time, I have found 5 common problems, that if not resolved, are very likely to cause a project to be a failure.

1. Not having a product owner

Of all the things that can cause an Agile software project to fail, not having a person that is ultimately the decision maker for the product being developed is the quickest way to ensure its demise.

It doesn’t matter if you are following Scrum, doing your own Kanban style project or something else; if you want your project to succeed, you need someone who can set its direction and make decisions about the product being developed.

Think about remodeling a kitchen. If you hired a bunch of contractors to come in and do the various jobs, like: plumbing; carpentry; flooring; etc, but you don’t have someone deciding what the actual finished kitchen should look like, you are going to get a random mess.

no product owner thumb 5 Things That Will Make Your Agile Development Project FAIL

A few skilled contractors will probably be smart enough to find the right person to ask about what should be done, but it takes more than just making arbitrary decisions about where the cabinets should go to design a kitchen. You need someone to actually come up with a real design and vision and be able to modify that vision appropriately as the project progresses and problems inevitably occur.

It seems pretty nuts to spend a huge amount of money remodeling your kitchen, but not want to invest any time or effort in either designing the finished product or hiring someone to do it.

Yet, day in and day out, I see this exact behavior in software projects. I see companies investing huge amounts of cash in Agile development, but not appointing anyone to be the true owner of what is being built and to set the vision for it.

2. Not iterating

One of the key values that Agile development brings to the software development world is iteration.

What is iteration though?

You may think it means breaking your project into 2 week sprints or iterations.  While doing this can facilitate iterative development, it doesn’t mean you are actually iterating.

Confused? Let me explain.

The key to iterating is to develop a product a little bit at time. It would be accurate to describe the process of iterating on a product as evolution of a product.

Whether you believe in macroevolution or not, microevolution, or adaptation is a proven concept.  The idea behind evolution is that things change gradually over time. Those small changes add up to result in a big change.

Imagine how silly it would be if evolution worked the way most “Agile” software is built.

Imagine, if you will, a fish that swims in the ocean and has some little fish babies that are born with fully functional legs. Then, those legged fish babies grow up and have fish babies that have wings.

evolution agile fish thumb 5 Things That Will Make Your Agile Development Project FAIL

Probably the legs and wings wouldn’t do that fish much good, nor would they be designed properly, because instead of adapting and changing over time, they just suddenly appeared.

Features should not be built in a single sprint or iteration. It is as silly as legs showing up on a fish in a single generation.

Instead, features should be evolved and grow over time. A feature shouldn’t be pushed into a single sprint or iteration and then be done. A feature should start out small and develop and evolve over time as feedback is received and customers or the product owner tries it out.

Far too many times, I see Agile development projects mistake having iterations with actually iterating the development of the product.

Don’t try to ship a completed feature at once, let it evolve over time.

3. Not breaking things down small enough

I am a big proponent of breaking things down into small, easily digestible pieces.

The main reason why this is so important is because it prevents procrastination. Procrastination usually occurs when either we dread some large task that will be difficult or we don’t know what to do next.

If you can break a big project up into small parts, it will seem easier to accomplish and will have clear steps of progression.

I often see software projects where the person creating the backlogs or work items is not considering the work enough before giving it to the team.

I coined a term for these kinds of backlogs: fatlogs. Fatlogs are backlogs that are not broken down small enough and often are very vague in what needs to be accomplished.

Fatlogs are notoriously difficult to estimate and waste time when trying to explain and understand them. It is critical that fatlogs be broken down into smaller actionable backlogs before being given to an Agile development team or a large amount of time will be wasted.

Many times, I have found that the creator of the fatlog could have easily broken down the work it represents into several smaller backlogs that would be easier to explain and understand, even without knowing much about software development.

I often recommend to Agile development teams that they outright reject fatlogs and send them back up the chain.

“If you can’t take enough time to clearly state what you want, it must not be that important.”

This doesn’t excuse development teams either. Development teams should break down any backlogs they get into small tasks as well.

4. Not setting done criteria

What is the first thing a waiter asks you when you order a steak?

That’s right, they ask you how you would like it done.

If the chef doesn’t know what the done criteria is for your steak, the chef will have to decide what he or she thinks the done criteria is.

done criteria agile steak thumb 5 Things That Will Make Your Agile Development Project FAIL

You may get back a well-done steak, or a rare steak, or something in between, depending on the personal preferences of the person cooking your meat.

This isn’t a good way to cook steaks and it isn’t a good way to cook software either.

In many software projects I often encounter lots of steaks being cooked but no one defining when they are done. The backlogs most often end up being “done” when time runs out.

It is very important to have explicit unambiguous done criteria for any backlogs being worked on by an Agile development team.

This means that the product owner should be defining some level of acceptance testing. It doesn’t matter if the tests are manual tests or fully automated tests (BATs), what matters is that some criteria is defined by which the team can evaluate whether they have achieved their goal or not.

Lacking this criteria is like a parent responding with “enough” when their child asks “how many more peas do I have to eat?”

Not having done criteria is demotivating and results in the finger pointing game of why the correct thing was not built.

I have found that if you tell someone exactly what is expected of them and what criteria they will be judged by, you get much better results than just throwing them in the ring, slapping them on the back, and saying “do a good job.”

5. Not letting the team be a team

Teams are strange organisms, especially Agile teams.

There are many dynamics that affect how teams act and interact. There are many ways to foster a healthy team and many ways to create unhealthy teams.

A healthy motivated team has synergy, a unhealthy gaunt team can be less effective than all of its members are on their own.

The key to having a healthy motivated team, is letting them be mostly autonomous. Regardless of your political persuasion, you would probably agree that when a country invades another country and sets up an unelected government to oversee the people, there are going to be problems.

The same thing happens in Agile development.

I’m not saying teams shouldn’t have accountability. But if you want to run a software project in an Agile way, you have to let teams self organize and self manage to some degree.

When big brother is always watching and interfering, team dynamics become very different. Naturally, teams usually develop their own cadence, leadership and roles (this is called norms.) However, when outside forces directly interfere with how the team does things, and they don’t have the power to decide their own fate, team members begin to realize that they need to look out for themselves.

Additional agile development resources

I’ve found it to be pretty difficult to find good resources on these topics, but here are a few books I’ve found useful that address some of the issues I described above.

  • Succeeding with Agile: Software Development Using Scrum - This book is focused on Scrum but it applies to many different kinds of Agile projects as well. Mike Cohn and I usually agree on most Agile subjects.
  • Agile Retrospectives - I’ve found this book to be useful for getting good idea for team retrospectives that can really help to break down some barriers and help teams learn to resolve issues themselves.
  • Scrumban and Kanban and Scrum - Both of these books have excellent information about combining Scrum and Kanban and have great solutions for dealing with some of the problems I described above.

What do you think? Have I missed any of the major Agile project killers? Any useful tips for dealing with them?

Let me know in the comments below.

7 Reasons Why You Should Tackle Hard Problems Last

I always hear the advice that we should tackle hard problems first.

It seems like pretty legitimate advice, and many of the reasons for saying so make sense, at least at a surface level.

The reasoning usually revolves around the idea that by tackling the hard problems first you can eliminate your biggest risk early and everything will be smooth sailing from there.

When you really think about the reasoning for solving hard problems first though, most of it is not actually reasoning, but based of off one emotion… fear.

We tend to think it is best to solve hard problems first, because we are thinking about eliminating our fear, not because we are thinking about what approach has the highest chance of success or is the most optimal.

I call this FDD, or Fear Driven Development.

And when I think about it that way, I find myself hard pressed to find a real good reason for tackling hard problems first besides being able to abort early.  Which, in some cases might be good, but I’d rather focus on success.

tackle thumb 7 Reasons Why You Should Tackle Hard Problems Last

Here are 7 reasons why it might be a good idea to tackle the hard problems last instead of first.

1. Solving easy problems first gives you momentum

When a large ball starts rolling down a hill, it picks up speed rapidly and that large ball can bust through many barriers that it couldn’t before, simply because of one thing it has after rolling down a hill that it didn’t have before—momentum.

On the converse, trying to push a heavy ball up a hill is pretty hard.  And if there are barriers on the way to the top of the hill, not only do you have to fight gravity, but you have to be able to push extra hard to get through those barriers.

momentum thumb 7 Reasons Why You Should Tackle Hard Problems Last

Life is hard enough, why make it harder?

I recently received an email from a developer that was concerned that his team wasn’t gelling and that they didn’t have the expertise in the technology needed to solve the complicated problem ahead of them.

They were going to start the project by trying to integrate this fairly complex technology and he was afraid that it would cause them a large delay before they would be able to get version 1 out.

My advice?

Start with your simple problems; get working software out there as soon as possible.  Not only will the team gel much more easily as they are having success and making real progress, but that momentum will help them when it is time to solve the more difficult problem. 

Even if they have to throw the first version away, when they get to the hard problem, the momentum alone will make them much more likely to reach success in the end.

I could give 100 examples of how solving easy problems to gain momentum can benefit you, but you probably already know intrinsically that this is true.

Long story short, get a running start before taking a big leap.

2. Avoid solving the wrong problem

wrong thumb 7 Reasons Why You Should Tackle Hard Problems LastThere are few things worse than spending weeks or months solving a difficult problem, only to find out in the end that you actually solved the wrong problem.

The big problem with solving the hard problems first is that the hard problems usually require a large amount of context in order to fully understand them.

It is very hard to get the right context for a hard problem when we take it out of its natural order of progression and artificially cut it to the front of the line.

You’d probably like to start a college class by taking the final exam first, so you don’t have to worry about it, but the problem with doing so is that you’ll lack the context and information to understand the questions and to know the answers.

When we try to tackle problems out of order to avoid leaving the hard problems to the end, we end up losing all of the learning and context that would help us to solve the hard problems at the end and we are much much more likely to end up solving the wrong problem, which is a complete waste of time.

3. Someone else may solve the problem for you

Sometimes procrastination is a good thing.

Sometimes, when you purposely push off solving a hard problem till the end, you end up finding that someone else already solved your problem.

editing thumb 7 Reasons Why You Should Tackle Hard Problems LastI was working on a Pluralsight video last week, using Camtasia 8 for editing, and I found that one of the video segments I was tried to load up was crashing the application every time.

I spent a few minutes trying to troubleshoot it, but nothing I was trying was working, so I had to make a decision.

I had 3 choices:

  1. Keep trying to solve this hard problem before moving on.
  2. Go on and do other videos and send off a support request to see if they could handle it.
  3. Make a new project and re-edit all the clips.

Choices 1 and 3 involved tackling a hard problem right then and there.

Choice 2 was to tackle easy problems and see if support could solve my hard problem for me, and if not, I would solve it at the end.

I ended up choosing option 2 and it paid off.  It turned out Camtasia support was able to solve my problem.  By the time I needed the project to complete my course, they had solved my hard problem for me and I didn’t waste any time upfront trying to tackle it myself.

Now it could have worked out differently; I might have had to solve the problem myself at the end, but instead of assuming I would have to and wasting perhaps a day or 2, trying to solve the problem myself, I pushed it off and kept working on easy problems and I gave someone else a chance to solve my hard problem for me.

It doesn’t happen all the time, but many times if we push off the hard problems we face, we find that by the time we get to them, someone else has already solved the problem for us.

4. Your own subconscious mind may solve the problem

When I said someone else might solve the problem for you, that someone else might actually by you—at least your subconscious mind.

Have you ever had the experience of thinking about a problem and not being able to figure it out, but then you wake up the next morning and suddenly have the answer?

It seems that our subconscious mind is more powerful than we are aware of.

sleeping thumb 7 Reasons Why You Should Tackle Hard Problems Last

In many cases, if we know of the hard problem we need to solve and have thought about it a little bit, our subconscious mind will start working on the solution, even though we are not aware.

Obviously this isn’t going to work all the time, and your subconscious mind isn’t going to write a bunch of code for you, but in many cases there is at least some benefit to throwing the problem off to our internal “worker thread.”

5. You are more committed to solving the hard problem when you risk everything you do so far

One benefit to saving the hard problem for last is that you have some extra motivation in the form of loss aversion.

It has been demonstrated in several experiments that people tend to try to avoid losses versus acquiring gains.

We can use this knowledge to our advantage by doing the easy work first and letting our loss aversion help motivate us to solve the harder problems, because we don’t want to lose all the work we put into a project so far.

By starting with easy problem, we put some “skin in the game.”

If we try to solve the hard problems first, we have nothing to lose, so we are much more likely to give up.

6. Hard problems are often easy problems bundled together

bundle thumb 7 Reasons Why You Should Tackle Hard Problems LastI’ve talked many times about breaking things down and trying to keep things as simple as possible.

And it turns out that many hard problems (not all) are decomposable into many small easy problems.

If you strive to never solve hard problems and to always push them off, you may actually find out that you never have to solve hard problems.

Many times we can chip away at hard problems, by taking bits of them off a little at a time and solving those easier problems.  Eventually, you may find that you’ve reached the tootsie roll center of your hard problem lollipop and it is filled with chocolate candy!

Now, some problems aren’t very easily decomposable, but a good majority of problems are.  Once you develop the skills to chip off bits of hard problems into smaller easy problems, the world looks like a much more conquerable place.

So, saving hard problems for last and breaking off little pieces of them as you go, can be a good strategy to help you to wear down your opponent before you have to face him.

7. Some hard problems are never truly solved

One of the big problems with tackling the hard problems first is that they tend to fill up as much time as you’ll give them.

If I give you an easy problem, like write a function to reverse a string, there isn’t much to think about.  You can solve it a number of different ways and there isn’t a huge difference in the efficiency of the different methods of solving it.  It is pretty unlikely you’ll spend weeks revamping your solution and thinking that it’s not quite right.

But, if I give you a problem like, create an in-memory database, not only is it a hard problem, but it has a huge number of possible solutions and can be optimized from now until the end of time.  You could spend 2 weeks working on that task or 2 years.

The problem is that many hard problems don’t have a good scope to them when they are solved in isolation.

If you design an engine for a car that isn’t built yet, you won’t know when you are done.

But if you design an engine for a car and you know how much it weighs and know what kind of transmission it will use and what kind of fuel efficiency it needs to have, you can have a much clearer understanding of when your engine design is done.

If you save the hard problems for last, the scope of those hard problems will be much more defined, which will keep you from wasting valuable time over solving a problem or, like I mentioned earlier, solving the wrong problem altogether.

If you like this post don’t forget to Follow @jsonmez or subscribe to my RSS feed.

Principles Are Timeless Best Practices Are Fads

There is a huge difference between a principle and a best practice.

Best practices are subjective and depend largely on context, while principles are eternal and universal.

time thumb Principles Are Timeless Best Practices Are Fads

After writing The More I Know The Less I Know, I received a few emails talking about how there are absolute best practices that should always be followed in software development.

I had already intended to write about principles, but that confusion made it clear to me that there should be a distinction made between best practices and principles.  We don’t want to throw the baby out with the bath water.

Looking at some examples of best practices

First let’s take a look at some software development best practices, then we’ll contrast them to principles to better get an idea of the difference.

One of the most common best practices today in software development is the idea of unit testing.  I’ve written about how I have my doubts about blindly following this best practice in the past, but whether or not we should follow it, is not what I am concerned with today.

Unit testing is extremely contextual.  What I mean by this, is that almost anyone would agree that there are a certain set of circumstances that makes unit testing have value.

If you work in an environment where the execution of unit tests takes a really long time, or you are developing your software in a waterfall approach where you have a big upfront design and detailed requirements, unit testing starts to lose value rapidly.

But rather than get trapped into the argument of when unit testing loses its value, it is better to address when it has the highest value—we are much more likely to agree there.

Unit testing has the highest value when we are working in agile environments where changes are being introduced into a software system rapidly and refactoring is taking place.  It also greatly increases in utility when you are able to execute and write the tests quickly, because that feedback loop makes it much easier to write the tests in a step by step approach, especially when doing TDD.

uml thumb Principles Are Timeless Best Practices Are FadsThere are plenty of other best practices that have fallen out of favor, like heavily commenting code and documenting requirements with UML diagrams, but context also greatly played a part in the value of these practices. 

When most developers wrote very short variable and method names, comments were really important.  Before Agile processes became prevalent, getting detailed requirements upfront was critical.

But, most best practices are good!

Yes, you are right, most best practices do apply pretty broadly and are generally helpful in a large number of different contexts.

For example, it is considered a best practice to use a source control system and it doesn’t seem like there are many situations where this wouldn’t be the case.

So doesn’t that make it a concrete rule or a principle?

No, it is still too specific to be generally applied in all cases and the act of putting your code in source control does nothing to improve the quality of your software or software product. 

If you were to blindly follow any best practice and not apply that best practice in a way that brings out the underlying principle, you would be very unlikely to actually receive any benefit.

You see, most best practices are actually derived from universally applicable principles that never change.  That is why most best practices are good.

The problem is applying the best practice itself in no way assures the benefit of its underlying principle.

To put it plainly, there is something greater at work that makes it a good idea to check your code into a source control system.  It is entirely possible to follow the action, but completely miss the spirit of the action.

More and more today, I see software development teams that are:

  • Writing unit tests
  • Using continuous integration systems
  • Using source control
  • Having Scrum meetings
  • Pair programming
  • Using IoC containers

Yet they are getting little to no benefit from it.  Just a bunch more pain and hoops to jump through.  The reason is simple…

It’s not the best practice that is effective, it is the principle behind the best practice

Principles are everywhere.  They apply in all aspects of our life.  You cannot go through the day without being affected by the results of 100s of different principles that have a constant influence on your life, just like the law of gravity does.gravity thumb Principles Are Timeless Best Practices Are Fads

Gravity is actually a great way to understand principles.  As far as we know, it is a universal force that is always in effect.  It is impossible to escape the law of gravity, wherever you go in the universe it affects you.

Principles are like laws of nature except bigger.  Principles are more like the laws of reality.  Even though you may not be able to describe them fully or understand how they work, they always work.

Take for instance, the law of the harvest.  Most people are familiar with this particular principle.  It basically goes like this.

You reap what you sow.

How universal is this truth?  How can anyone avoid it?  How many times have you found yourself subject to this inescapable law about how reality works?

Many software development best practices are actually based on this principle.  Think about best practices that have you make efforts to improve the quality of software early on in the process.

TDD or test driven development, is such a best practice.  The basis of TDD is to introduce quality into the software development process as early as possible, so that the finished product is better.

If you apply the practice of TDD without understanding this principle, you are just following the motions and you won’t actually gain the benefit of the practice.

If you can’t understand at some level that the point of doing TDD is to sow some good seeds in your software that you will harvest later on, you won’t be writing the right kind of tests.

There is nothing magical about writing tests before writing code, but there is something valuable in purposely investing in upfront quality with the end goal of getting a big yield on that investment in the right season.

By the way, that is why I like Bob Martin’s book Agile Principles, Patterns and Practices in C#; it discusses many principles of software development that are timeless.  Books like this one and the book I have mentioned probably 10 times in this blog, How to Win Friends and Influence People, are full of principles.

Also, check it out, you just learned what Agile really is.  With principles in mind, now read the Agile manifesto.  It was never designed to be a detailed process and set of best practices for developing software, it was always meant to be a recognition of a set of principles that guide software development.

So, just remember the next time you are arguing with someone over a best practice, or consider applying one to a project you are working on, if you don’t understand the underlying principle, no amount of ceremony and procedure will have the smallest amount of benefit.

If you like this post don’t forget to Follow @jsonmez or subscribe to my RSS feed.

My End of the World Post

With the coming and passing of the predicated date of the Mayan apocalypse, I got to thinking about what kind of final advice I would leave the development world in the event of my untimely departure.

I suppose this is quite a strange thing to ponder given the circumstances, since if the world did indeed end there wouldn’t really be anyone to pass this advice onto anyway.  Also, given that I am writing this blog post now, it would already be too late.

Regardless, it got me thinking about the little things I have learned in my development career that might be helpful to others.

So without further adieu, if I could only leave one blog post to the future generation of developers, here is what it would be.

1. Break things into small tasks

This might seem like a strange thing to put at number one.

break My End of the World Post

I mean really, of all the advice that you can give, breaking things into small tasks is number 1?

Yes, it is that important!

I’ve written about how smaller is better and how breaking even small things up is important, but I’ll hit on this point one more time from a slightly different perspective.

Breaking things into small tasks is not just a skill that will make you a better developer, but it is a critical thing that will determine whether or not you enjoy what you are doing.

I have found so many times that when I have become unhappy with my work or felt distraught by the overwhelming task ahead of me, it is mostly because I have failed to break down a chunk of work into small tasks.

Just this last week I re-realized how much a difference just breaking things up into small enough tasks can make on the overall morale of an entire team and my individual feel of satisfaction with the work I am doing.

Time and time I come back to this very real realization; that breaking things up into small tasks doesn’t necessarily make me a better developer, but it makes me a much happier developer.  (Although it is quite easy to make the argument that it makes me better at my job as well.)

2. Help others

This one may also come as a surprise, but I’ve found that helping others is one of the best ways to help yourself.

help My End of the World Post

Not that we should have selfish ambitions when helping others, but I want to point out something that many developers don’t realize when they are first starting out.  Most developers never realize how much they actually benefit from helping other developers.  I have been in plenty of great environments where every developer was willing to help anyone else who needed their help, but I have also encountered many environments where that was not the case.

Sometimes we can be trapped into thinking that helping others will reduce our own value, because we will be giving out our valuable knowledge to others for free.

This way of thinking could not be further from the truth.  In contrast, I have found that helping others results in a much deeper understanding of things yourself.  I’ve talked about how teaching something is really the best way to learn a subject, but it goes much deeper than that.  When someone asks questions about something you take for granted, you have to answer a question you don’t normally ask yourself when you know how–why.

Helping others forces you to answer the question of why you are doing something a certain way and brings to the surface of your mind a much fuller understanding of a problem.

In addition to gaining deeper understandings of what you already know, helping others often will expose you rapidly to a large number of problems.  I cannot express how valuable the increased exposure to problems is in accelerating your own learning.

3. Speak up tactfully

Perhaps you may not feel like what you have to say is important or that you don’t want to cause conflict or maybe you are just trying to be agreeable, but whatever your reason is, “stop it!”

speakup My End of the World Post

You need to express your voice, because doing so is going to actually make you part of the team rather than being a passenger on the bus.

I often see that many developers either remain quiet or speak their mind with no temperance in what they say.

It is very important to say what you think, but it is just as important to do so in a tactful way.  I realize this is much easier said that done, but mastering this skill will help you immensely in your career.  I have found that very rarely will someone be offended by your ideas, even if they contrast their own, if you present your ideas in a tactful way and you are willing to change directions when you realize you are wrong.

4. Automate

It is amazing how many things many of us do over and over again each and every day and we never take the time to automate them.  The more brainless tasks that we do which could be automated, the less time we have to do things that actually improve our skills and really matter.

I’ve found that I often fall into the trap of thinking that I don’t have enough time to automate tasks that I am doing, so I keep doing them over and over again.

I still have about a dozen things I can think of right now that need to be automated.

As software developers we should really take the time to automate these things, because software development is automation.

5. Understand first

Another major trap that we can fall into is trying to jump into things without fully understanding the problem.

I don’t know how many times I have tried to fix a bug without actually reproducing it first.  Many times I prematurely jump into trying to solve a problem before I fully understand it, because I feel like I don’t have the time to spend learning about the problem.

This is always a wrong choice!

I’ve found time and time again that the moment you fully understand a problem you solve it. (There are some exceptions of course.)

6. Have a side project

Not only will it likely provide you a second source of income, but it will improve your skills and give you an important outlet to learn and explore without the outside pressure of time or politics.

It can be pretty difficult to have a full time job and still try and do something on the side.  I know that after working for 8 hours the last thing you probably want to do is come home and work some more, but there is a very large value in doing something for yourself that adds up over time.

Most of us spend our day jobs working for someone else and making them money.  Most of our day jobs involve doing work that only benefits us for a single paycheck.  What I mean by this is that the work you do at your job doesn’t continue to pay you dividends over time.  So if you work your regular job for 20 years at the end of 20 years, the only thing you will have is the money you earned from that work.

Now think about what happens if you devote 1 or 2 hours a day to doing some work for yourself every day as well.  1 or 2 hours a day might not seem like much, but over just 2 years that is 500 to 1000 hours of work on a side project.  Think about what you could accomplish in 500 to 1000 hours.

7. Read books

I’ve talked about this idea many times, because it is so important.  You should always be reading some kind of book that will improve your skills and knowledge.  I do videos for Pluralsight, and videos and blog posts are great, but books are still vitally important.

read My End of the World Post

Books require much more thought and are processed much more than any other medium.

Think about what goes into the process of writing a book.  Books are outlined, then written, edited and revised and technically proofed.  Books also tend to cover subjects much more deeply than a blog post or a video series.

So, I am not saying to avoid other mediums, but I am saying that quality books are an important part of improving your abilities as a software developer.

8. Do the right thing

Bob Martin had an great post on saying “no.”  The basic idea is that you are a professional and because you are a professional sometimes you’ll have to basically disobey what your client or employer asks for and instead insist on doing the right thing.

I recognize there is often a great cost associated with this line of thinking– you might lose your job– but if you ever want to consider yourself a professional, you have to be able to make these kinds of tough choices.

In my experience, I have never been fired or even detrimentally affected by taking this stance.

I’ve always tried to take this position in a very respectful way and that has probably been a large reason of why I have had success with it.  I have found that most employers and clients will respect you and value you more even if they are a bit ticked off initially.  This goes back to the idea of speaking up and doing so in a tactful way.  When you do this correctly and you insist on doing the right thing it is called integrity and even if someone disagrees with your position, most people will respect you for it.

9. Be kind

Seriously, is there any benefit to you or anyone else by being mean?

I know it isn’t always easy to be kind, but you should really strive to be, because it does no good to treat people poorly.

Alvin the Chipmunk once said that you’ll meet the same little people on the way down as you did on the way up.

I can’t make an effective argument of how being kind to others will benefit you, and even if I could I wouldn’t try, because that is really not the point.  The point is we are all human beings and we are all deeply flawed, but every single one of us has value and that value can be appreciated.

So, just be nice to everyone you meet along your way and you’ll have a much better time of it.  Call it Karma if you will.