Category Archives: Math

The Myth of the Super Programmer

I received an email this past week that disturbed me.

Basically the author of the email inferred that most of the topics I talk about in my blog posts and Pluralsight videos are relatively easy topics, but that I had hypocritically suggested that interviews should be hard and should be designed for “real programmers” or super programmers.

Essentially the point of the email was that application developers are not “real programmers” and “real programmers” do hard stuff with difficult math.

Searching for the super programmer

super duper awesome programmer thumb The Myth of the Super Programmer

I don’t think this attitude or understanding about programming and software development is unique or irregular.  Even Scott Hanselman has called himself and thought of himself as a phony.

His post on the subject resonated with me, because I think the same thing about myself sometimes.

Sometimes, I wonder if I could really tackle the really hard problems.

I would venture to say that most software developers have some sort of belief that they are just a regular programmer, but there exists out there some super programmers who actually do the difficult algorithms that control caches on hard drives and index search results for Google.

Now, of course there are programmers writing code that does all kinds of complex things that you and I don’t understand, but how different are those programmers from the rest of us?

Is there really a difference between an enterprise application developer and a programmer who writes search algorithms for Google and firmware which controls the mechanical operation of the head of a hard drive as it reads data from the platter?

Before I give you my answer…

Let’s talk about problems for a minute.

What is the most difficult problem you have ever been asked to solve?

How did you go about solving it?

In the end, when you actually solved the problem, did the solution seem easy?

When you go back and look at the problem, does it seem much simpler now?

Lots of questions, I know—but I want you to really take the time to think about those questions before reading on.

It is important to understand the difference between perception and reality and many software developers, myself included often have trouble distinguishing between the two.

You see, our perception of a problem is often much different than the reality of that problem.  When we don’t understand a thing, it seems much much more complex than it is.  But, once we come to understand a problem, we see how simple it actually was to begin with.

Let me give you a real example.  Take a look at this mathematical equation below.

e1 thumb The Myth of the Super Programmer

Now there are two kinds of people who will look at this equation.

  1. Those who have a decent understanding of mid to advanced mathematics and immediately recognize it and understand it instantly.
  2. Those who have never seen these symbols used together and immediately think this is some kind of complex thing that would take them years to learn.

I may not be exactly correct, but my point is there is a clear division between those who understand and those who don’t.

I can very simply explain this set of symbols to you in terms you already understand.

Ready?

That equation is the same as this code:

var total = 0;
for(int i = n; i <= m; i++)
{
   total +=  f(i)
}

So what is my point?

My point is that there are very few actual hard problems in math, programming, heck life in general, and usually those few hard problems can be decomposed into smaller problems (sometimes more than once), until you end up with a simple problem.

The point of this blog, the point of my Pluralsight videos, the point of my life, basically, is taking complex things and making them simple.

If you want to succeed as a programmer, you have to learn how to do this for yourself, because this is the single most important skill you can possess.

So, to answer the original question—No, I don’t believe there are super programmers.  I don’t believe that there is a difference between an enterprise application developer and programmers working on problems that most other programmers would perceive as really hard problems or “real programming.”

Now, don’t get me wrong and think that I am saying that I don’t think some programmers are orders of magnitude better than others.  I would venture to say that really good programmers are about 10 to 20 times as effective as just average ones.

What I am saying though is that we have a tendency to forget how simple all problems really are when they are decomposed to smaller problems and that ALL problems can be decomposed in that way.

What I am saying is the thing that might be stopping you from growing to become a really good programmer is your own false belief that you can’t possibly understand something that you currently perceive to be complex.

What I am saying is that when you write what appears to you to be a simple enterprise application, you might be forgetting just how difficult and damn near impossible it seems to all your friends and family that know nothing about programming.

Still think I am wrong?

Fine, you are entitled to think so.

But I do have a challenge for you.  Surely you know a “super programmer.”  You might even be one.  If so, let’s hear from you or them.  Tell us that complex problem that is too difficult for the rest of us to possibly understand.

I don’t even mean that sarcastically at all.  Seriously, if you can prove me wrong, do so.  I’ve just never seen a problem that couldn’t be broken down into simple to understand components, yet.

Solving Problems, You Better Learn How

It is astounding how many developers can write and maintain large enterprise systems dealing with all kinds of complex logic, database access, etc and cannot for the life of them solve a moderately difficult programming problem given in an interview in less than 30 minutes.

It is also astounding how many developers that can not write even a single line of code can fail the same problem exactly the same way.

Based on those two statements, your first assumption might be that asking someone to solve a programming problem as part of an interview would be a bad idea, because it isn’t really telling you if they are good and just crumble when asked to do something like this on the spot, or if they can’t program at all.

thinker1 composition5 small2 thumb Solving Problems, You Better Learn How

Let’s look at some possible numbers to see why this is wrong

In poker you can use a combination of pot odds and the likely holdings of your opponents to deduce in many situations whether or not you should call a bet.

Pot odds is simply a ratio of how much money is in the pot (what you can win), versus how much it will cost you to call the bet.

For example, if a pot contained $9 and I bet you $1, you would be getting 10:1 odds on your money for a call (you have to risk an additional dollar to potentially win 10.)

So in that example, if you have a better than 10% chance of winning, you are making a profitable play by calling.

I don’t have the exact numbers (actually I am just guessing at them from experience), but here is a realistic estimate:

  • Good programmers – 60% can solve the problems
  • Great programmers – 90% can solve the problems
  • Don’t know how to program, lied on my resume – 1% can solve the problems by some sort of divination or black magic, or memorizing sequences of symbols which solve the problem.

Given some statistics like this, if you have someone pass this kind of a test, you can calculate the rough probability of which of the three groups they belong to.

  • 40% chance they are Good
  • 59.5% chance they are Great
  • .5% chance they are a black magic practicing witch or warlock
  • 99.5% chance they are either good or great

So, even though you may be throwing away some good candidates and some great ones, the chance of you getting someone who doesn’t know how to code at all is almost reduced to nothing.  This is very important!

Why?  Because if you hire someone who doesn’t know how to code at all, it will cost you a huge amount of money, but if you pass over someone who is great, but still end up getting someone else who is good or great it doesn’t really cost you anything except in the rare case that the great programmer failed out of the test and you ended up hiring a good one instead.

Even still, that is a small loss versus hiring someone who can’t write a lick of code.

I’ve talked about how I think having code in an interview is a good thing before, but now the numbers are here to back me up.

More and companies are realizing the truth about this situation and companies like Codility are popping up to offer a service to test candidates for you.

What this means for you?

You might not get asked to solve a problem or write code in your next interview, but there is a growing chance that you will.

You may think to yourself, ok even if I bomb the code writing part, I will do great on the rest of the interview and be able to convince them that I know how to write code, besides any company that would toss me out of the running for failing some stupid online test is a company I don’t want to work for anyway.

To that I say a company that doesn’t know how to do math is a company that you don’t want to work for. If my numbers are anything close to accurate, the mathematical best choice is to almost always throw out someone who fails the coding test.

The simple solution is to learn how to solve these kinds of problems.  Learn how to write an algorithm under pressure in 30 minutes.  It’s not that hard to practice and the skill will help you in many other areas of your career and probably your life in general.

I’ve written before on TopCoder as a great site to practice these kinds of problems, and if you haven’t checked it out that is a great place to start.

Another option is to get the book Programming Pearls.  It is a bit dated, but has some nice programming problems for you to try to solve.

I’ll follow up this post in the coming weeks with some hints and tips on how to take a programming problem, break it down and finally code it.  Learning how to do this properly can make some of the more difficult algorithm type problems very simple.

So get to it!  Start learning how to solve programming problems.  Reverse strings, sort linked lists, help grandma pick the highest yielding grandsons and a granddaughters to send Christmas cards to!

If you are following my back to basics series, don’t worry, it is still on.  I’ll probably pick back up with it after the holidays.

As always, you can subscribe to this RSS feed to follow my posts on Making the Complex Simple.  Feel free to check out ElegantCode.com where I post about the topic of writing elegant code about once a week.  Also, you can follow me on twitter here.

Importunate Permutations

Here is an interesting programming problem: Calculate all the permutations of a string.

For example, the permutations of “abc” are:

abc

acb

bac

bca

cab

cba

It is not as easy of a problem as it seems, but it has a rather simple solution.

Many times recursion is actually more complex to understand than a non-recursive solution, but in this case recursion is an elegant and simple solution.

A good way to solve these kinds of problems is to start with N, N+1, N+2, then solve for N+x.

Let’s start with N; we will define a function permutation which will give us the permutations of any string.

permutation(“a”) =

“a”

permutation(“ab”) =

“a” + permutation(“b”)

“b” + permutation(“a”)

permutation(“abc”) =

“a” + permutation(“bc”)

“b” + permutation(“ac”)

“c” + permutation(“ab”)

permutation(x)

foreach character c in x

c + permutation(x remove c)

Looking at it in C# code, it would look something like:

public static List<string> Permutations(string s)
{
      if (s.Length == 1)
      {
            return new List<string> { s };
      }

      List<string> permutations = new List<string>();

      foreach (char c in s)
      {
            string leftOver = s.Replace(c.ToString, "");
            List<string> stringsReturned = Permutations(leftOver);
            foreach (string permutation in stringsReturned)
            {
                 permutations.Add(c + permutation);
            }
       }
       return permutations;

}

Update: Here is a pretty slick example in LINQ that I got from a stackoverflow.com question.

public static IEnumerable<string> GetPermutations(string s)
{
    if (s.Count() > 1)
        return from ch in s
               from permutation in GetPermutations(s.Remove(s.IndexOf(ch), 1))
               select string.Format("{0}{1}", ch, permutation);

    else
        return new string[] { s };
}

For a better understanding of permutations, check out this short booklet on How to Understand Permutations and Computations .