In my last post in the Back to Basics series, I mentioned that I would begin this series with the intent of showing you step by step how to go from no automation to a quality BAT infrastructure.

By the end of this series you should have the information you need to actually get automated testing going in your organization instead of just talking about it.

First stop on this train is hiring an automation lead.

Don’t doom yourself before you start

If you think you are going to create BATs, but you are not going to have at least 1 dedicated resource to doing the job of getting this all going, you are probably bound for failure.

Listen up, because this is the single biggest mistake that most organizations make when trying to get BATs up.

This mistake is thinking that you can get your QA team to get this going and you can part time some developers onto the project, but not having a dedicated resource.

Why is this absolutely absurd and you should expect it to fail?  Because, in order to get an infrastructure to support your BATs, you are going to need to basically design and create a testing framework that is tailored to the business domain of your application.

Blah, what?  Ok, in simple terms.  Someone is going to need to design and build an automation framework.

If you think you are just going to buy some tool and use it to record test scripts, you are wrong.  That approach simply is not going to work.  I won’t get into why here, but I have written on it before in the past, and if you are interested, let me know and I’ll go much more in depth.

Think about this, do you want your QA team, as good as they may be at QA work, designing and building an automation framework?  Give them a few C# or Java classes and have their first real coding work be to design a complex system?

How about a developer who has “real” work and can only devote part time to designing what may end up becoming one of the most important parts of your infrastructure?

It is fine to pull a developer off of the main product and put them in this role until it is up and running, but make no mistake, you need a dedicated resource.

You are going to want to a well designed maintainable automation framework, because the ease of use of that framework will determine how easy or not it is to write automated tests that use that framework.

I recommend getting someone who has done it before

Why?  Because creating an automation framework is much different than developing your software and it is much different than writing tests.  It really requires a special skill set that is not so easily acquired without the experience of having done this before.

I also think it is fine to use a developer resource, but if you do, you want to try to get a more senior developer to take this role.

Here is a list of some of the skills that I think are important which may also make it much more clear why I am recommending a specialist in this case:

Development of APIs (In the end the automation framework will be an API that is used to write automated tests.  The better the API is designed, the easier it will be to write tests against it.)

Design of reusable base components (Much of the framework consists of repeated concepts with different variations for different screens in your application.  As a result, the core of the framework will be a basic design that allows for variation without too much repetition… reuse)

Ability to quickly acquire domain knowledge of the software (Someone writing an automation framework needs to really understand how the system under test works, so that they can model the framework after it.)

Pragmatism (Many “accepted” good software designs are actually bad automation framework designs.  Static methods can make an automation framework’s API much more simple to use, especially for non-programmers, but would not be a good design choice in your main product.  A person who designs an automation framework should be pragmatic enough to “break the rules” in order to make the framework easiest to use and maintain.)

Experience or understanding of QA (It is important to understand the users of the framework, the QA team.  Understanding what kind of tests they will likely write or how they will think will go a long way in coming up with a good framework design.)

Communication skills (Good BATs read like English.  It is critical to make sure that the person designing the framework facilitates this to make the BATs as useful and understandable as possible.)

HTML / XPath (This specific skill set is very important for web based products.  Creating an automation framework built on top of one of the web browser drivers will require intimate knowledge of HTML and probably XPath to be able to access attributes and elements on pages in your application.)

I’ve written a few automation frameworks and contributed to the design of several others, and I can tell you that my version 6.0 of an automation framework was much much better than version 1.0.

There are a huge number of patterns that emerge that are unique to this problem domain.  So, I do believe having experience in this area specially is a huge benefit.

What does this automation person do?

We’ll get to more specific in my later posts in this series, but the basic responsibility should be:

• Figure out the technology and tools to be used
• Create the initial automation framework
• Create smoke tests for the application under test
• Get those smoke tests into some kind of automated build
• Train developers and QA in the use of the framework
• Drive the adoption of the framework and automation for all features
• Set up rules for consistency in writing automated tests
• Design of hierarchy and structure of automated tests

You want your automation person to be responsible for pushing and driving forward the goal of getting to the point of being able to automate each feature being created with minimal effort.

This is one of those jobs that can actually be “done.”  What I mean by that is you may have the automation lead end up creating the framework, developing it enough to push it out to your development teams and train them to write automated tests along with QA, and you really don’t need an automation lead anymore.

This is great though, because now you have a developer that really knows your product and is a good bridge to QA, or you can keep them improving the framework and overseeing it if you want to go that route.

Bottom line is:

1. Make sure you get a dedicated resource
2. Make sure that resource can write code and design a framework
3. Make sure they train everyone else, one person cannot automate your entire application

As always, you can subscribe to this RSS feed to follow my posts on Making the Complex Simple.  Feel free to check out ElegantCode.com where I post about the topic of writing elegant code about once a week.  Also, you can follow me on twitter here.

It is also astounding how many developers that can not write even a single line of code can fail the same problem exactly the same way.

Based on those two statements, your first assumption might be that asking someone to solve a programming problem as part of an interview would be a bad idea, because it isn’t really telling you if they are good and just crumble when asked to do something like this on the spot, or if they can’t program at all.

Let’s look at some possible numbers to see why this is wrong

In poker you can use a combination of pot odds and the likely holdings of your opponents to deduce in many situations whether or not you should call a bet.

Pot odds is simply a ratio of how much money is in the pot (what you can win), versus how much it will cost you to call the bet.

For example, if a pot contained $9 and I bet you $1, you would be getting 10:1 odds on your money for a call (you have to risk an additional dollar to potentially win 10.)

So in that example, if you have a better than 10% chance of winning, you are making a profitable play by calling.

I don’t have the exact numbers (actually I am just guessing at them from experience), but here is a realistic estimate:

• Good programmers – 60% can solve the problems
• Great programmers – 90% can solve the problems
• Don’t know how to program, lied on my resume – 1% can solve the problems by some sort of divination or black magic, or memorizing sequences of symbols which solve the problem.

Given some statistics like this, if you have someone pass this kind of a test, you can calculate the rough probability of which of the three groups they belong to.

• 40% chance they are Good
• 59.5% chance they are Great
• .5% chance they are a black magic practicing witch or warlock
• 99.5% chance they are either good or great

So, even though you may be throwing away some good candidates and some great ones, the chance of you getting someone who doesn’t know how to code at all is almost reduced to nothing.  This is very important!

Why?  Because if you hire someone who doesn’t know how to code at all, it will cost you a huge amount of money, but if you pass over someone who is great, but still end up getting someone else who is good or great it doesn’t really cost you anything except in the rare case that the great programmer failed out of the test and you ended up hiring a good one instead.

Even still, that is a small loss versus hiring someone who can’t write a lick of code.

I’ve talked about how I think having code in an interview is a good thing before, but now the numbers are here to back me up.

More and companies are realizing the truth about this situation and companies like Codility are popping up to offer a service to test candidates for you.

What this means for you?

You might not get asked to solve a problem or write code in your next interview, but there is a growing chance that you will.

You may think to yourself, ok even if I bomb the code writing part, I will do great on the rest of the interview and be able to convince them that I know how to write code, besides any company that would toss me out of the running for failing some stupid online test is a company I don’t want to work for anyway.

To that I say a company that doesn’t know how to do math is a company that you don’t want to work for. If my numbers are anything close to accurate, the mathematical best choice is to almost always throw out someone who fails the coding test.

The simple solution is to learn how to solve these kinds of problems.  Learn how to write an algorithm under pressure in 30 minutes.  It’s not that hard to practice and the skill will help you in many other areas of your career and probably your life in general.

I’ve written before on TopCoder as a great site to practice these kinds of problems, and if you haven’t checked it out that is a great place to start.

Another option is to get the book Programming Pearls.  It is a bit dated, but has some nice programming problems for you to try to solve.

I’ll follow up this post in the coming weeks with some hints and tips on how to take a programming problem, break it down and finally code it.  Learning how to do this properly can make some of the more difficult algorithm type problems very simple.

So get to it!  Start learning how to solve programming problems.  Reverse strings, sort linked lists, help grandma pick the highest yielding grandsons and a granddaughters to send Christmas cards to!

If you are following my back to basics series, don’t worry, it is still on.  I’ll probably pick back up with it after the holidays.

As always, you can subscribe to this RSS feed to follow my posts on Making the Complex Simple.  Feel free to check out ElegantCode.com where I post about the topic of writing elegant code about once a week.  Also, you can follow me on twitter here.

This is a very simple problem that can easily be solved, yet not too many hiring managers or developers responsible for doing interviews are willing to do it.

All you have to do is increase the technical difficulty of the interview.

But I don’t want to ask hard questions, about a specific technology

Often I hear the point made that a good programmer can learn any framework or language, so asking difficult C# questions or asking details about EJBs will exclude those candidates prematurely.

While this statement is partially true, I don’t consider it a very good excuse for not asking hard questions.

There are many difficult subject matters in software engineering that exist beyond the domain of a specific language or technology.

You can always ask hard questions about object oriented design, design patterns, or general language constructs.

If you have the resources, you can always tailor the interview questions to the specific skill sets the candidate has indicated on their resume.

The truth is a really good programmer is going to know a lot about a lot of different areas.  A great programmer should also know a lot about some specific areas and languages.

What if all the candidates fail?

They all should fail.

Your technical interview should get increasingly harder, until it ultimately results in stumping the candidate.

Make your questions progressively more difficult until you reach an area beyond the knowledge of anyone on your team.  If you ever have a candidate that makes it that far, you know they are an instant hire.

You expect that no one will make it all the way through, and that is good.  It gives you a very real and easy comparison gauge on the different candidates.

Not that you should base your decision on how many interview questions the candidate gets right, but it can definitely provide you a comparison of two candidates that you wouldn’t be able to see without escalating difficulty questions.

For example, if candidate A can only get through the basic questions and candidate B makes it to the expert level questions, that should give you a pretty good idea of their relative talents.

Besides, failing is good.  We all fail every day.  I always want to see what a person is going to do when they don’t know the answer.  Do they have the humility to ask you the answer so that they will know it in the future?  Do they bring up that they will research that area?  Do they blow up and make excuses or try to justify a wrong answer?

I hate hard interviews, so I don’t want to give them out

Do you really hate difficult interviews?

Or do you hate pointlessly difficult interviews?  There is a huge difference.

Asking someone to use bit manipulation to reverse a string is stupid and pointless.

In most cases, asking someone to implement a quick sort algorithm is also stupid and pointless.  (Any programmer worth his salt would never trust himself to implement a well known sorting algorithm, and would instead look it up.)

My point is that there is a big difference between asking hard questions and good hard questions.  If you ask relevant hard questions, you are going to quickly turn away the “fakers” and “con artists” and you are actually going to make a good programmer pretty happy.

Nothing I hate worse than walking out of an interview, as the interviewee, feeling like the interviewer didn’t ask me hard enough questions.

I feel like if an interviewer asks me easy questions that anyone should know, I don’t get to demonstrate my real skills and talent.  When I leave an interview, I want the interviewer to think.  “Damn, that guy got almost every single hard question right, no one else was able to do that.”

So in reality, if you ask good hard questions, people won’t resent you for it, they will appreciate it.  It is nice to “slam dunk” a difficult interview, because you know that you likely have the job.  It is not nice to “slam dunk” an easy interview, because it doesn’t give you any confidence that your skillset was noticed at all.

Don’t forget to have them write code

Programmers should be able to write code.  In a high stress situation, it might not be pretty, and it might not be absolutely semantically correct, but it should be obvious that the candidate knows how to write code.

The best approach I found is to use a simple programming problem.  A problem that anyone should be able to solve, but can be solved many different ways.

The idea is that the code should be what you are looking for, not if they can solve a difficult logic problem.

I’m pretty happy to say that the company where I work now, TrackAbout, has a pretty difficult interview process which does involve writing code.  I am very happy about this, because I feel that it allows us to hire the best programmer, which allows me to work with the best programmers.

If your company is taking the easy way out of doing technical interviews, you are basically playing the programmer lottery.  It is definitely worth a little more time to ask harder questions and be a lot more confident in your decisions.

If you drive your car every day, do you think you are becoming a better driver?

Now, let me ask you another question.

If you competed in races with your car, frequently, do you think you would be becoming a better driver?

Let’s reflect that paradigm at programming.  Many of us, if we are fortunate, get to write code every day.  Unfortunately, a large amount of the code we write is fairly mundane.  We are rarely writing code in any form of competition.  When you are first starting out , writing code makes you better at writing code.  After time, when you have been writing code for a while, just like driving, you stop making gains.

You may have heard someone say there is a difference between a programmer with 10 years of experience and a programmer with 1 year of experience 10 times.

How to start making gains again

What I want to talk about today is just one of the ways you can start making gains again in your programming skills.  There are many others, and I have talked about some of them before.  But, today I want to talk about a very specific way I have found to break through the barriers and really improve your programming skills.  Specifically, I want to talk to you about a programming competition that I really enjoy called TopCoder.

TopCoder is a website that has a nice little competition arena that allows you to compete against other programmers solving some pretty difficult problems.  They have built a web launcher that opens up an IDE where you can view a programming problem and write your code to solve it.  You can solve the problem in any one of their supported languages and get immediate feedback through their system tests.  You can also view other people’s solutions to the problem and challenge their solutions with your own input.

There are other kinds of competitions on the site, but for what I am talking about you’ll want to go to the site, then click on develop, then click on algorithm, and launch arena.

Once you are in the competition arena, you are brought to a chat lobby, where you can switch to a competition room (if one is currently underway), or to a practice room, where you can practice some of the past competitions.

Once you are in one of the rooms that has problems, you can select a problem to launch the IDE.

There you can type your solution to the problem right there and compile or test your code.  I’m not going to go into all of the details of how the competition and scoring works here, because I want to focus more one why TopCoder, and other types of training like this are so beneficial.

Why will this make me a better programmer?

It is kind of hard to explain why this works so well.  I think one of the major reasons is because we are challenged to solve much more difficult problems than what we usually solve in our day jobs.  When you start solving those problems, the easier problems you encounter on a day-to-day basis seem to get a lot easier.

One thing that really helped me was the ability to see other people’s solutions to the problem.  When I would finish coding my problem, and look at the other solutions to the problem, I would always find new ways of using some of the base language or library features that had never occurred to me.

When you try to challenge someone else’s problem, you are forced to read some code that can be very obscure and quickly understand it.

You can also use some plugins to code in your favorite IDE and then have the code mirrored in the TopCoder IDE, but there is a nice gain in mastery of the language that can be had if you choose to use the TopCoder IDE which does not have auto-complete.  (Especially if you ever have to write code on a white-board during an interview.  It can be pretty embarrassing when you are unable to produce the syntax for a simple statement because you are so used to relying on auto-complete.)

You don’t have to do the competition, you can just do the practice rooms on the site, but I encourage you to try a competition or two.  There is just something about competition that seems to make us better.

Post me some comments if you try this and it helps you or hit me up on twitter @jsonmez.

There is a right or good way to market yourself.

Selling yourself tastefully takes some practice and thought.  It really is a soft skill that must be acquired and thought about.  It takes some amount of social engineering.  Like all products, the better the product, the easier it is to market.  Here are some tips I find useful for marketing yourself as a professional developer.

1. Create a weekly report of exactly what work you did each day of the week. The report should include a brief description of the major and minor accomplishments each day and perhaps a summary or highlight of any major accomplishments you did.  Especially mention mentoring or helping other people.  Send the weekly report voluntarily to your boss and CC your boss’s boss as an FYI.  I know it’s a bit difficult to find a good way to CC your boss’s boss, but try to find a good way to do it that is tasteful.  (For example, something like “I thought you might also like to receive these weekly reports I am sending out just as a FYI.” )  The benefit of doing this is two-fold.  First, you will be highlighting all the good work you are doing to your boss and his boss, in a way that isn’t like “tooting your own horn.”  Second, you will provide yourself a documented history of work you accomplished which is useful for reviews, protection against wrongful termination, and resume building.

2. Update your resumé periodically, and your public profiles, LinkedIn etc. By keeping your resumé constantly up-to-date, you are ready to find a job quickly if you become unemployed.  This will give you a safety net, which will allow you to feel more comfortable being a professional and sometimes saying “no”.  In addition, opportunities will come to you when you have your public profiles up-to-date.

3. Get a library of books and put them up for display. (Read them, or you are just faking it.)  If you have a mountain of books you have read, but they are sitting in a box at home, they are not marketing you.  When I walk into a developer’s office and I look at his or her books I can tell many things about them already.  When managers walk by and they see someone with a large number of programming books, it says something to them about your dedication and knowledge.

4. If you have certifications frame them and put them on your wall, or on your cube. Yeah, I know it may seem cheesy, and you may get poked fun at a bit.  But take it light-hearted and don’t brag about the certification, deep down inside the people poking jabs about certs really are feeling threatened they don’t have certifications.

5. Help other developers, all the time. Be they guy that everyone asks their development questions to.  When someone needs help figuring something out, volunteer.  Help train the junior developers.  Doing this will give you a reputation of being knowledgeable and helpful.  Answering questions on Stackoverflow counts towards this if you have time.

6. Prepare some power points and offer to give demonstrations or presentations on a project you did, or a new technology or methodology you want to introduce.

7. Be vocal. Make sure that you are speaking up.  In meetings, in design sessions, in general have something to say.  Make sure you are not talking for the sake of talking, but contribute information and ideas.  Doing this will help demonstrate your knowledge and critical thinking.

8. Make occasional visits to the boss. Make sure you are being seen by management every once in a while and when you do, mention new ideas you have of ways to make your team better or more efficient.  Offer to spearhead the implementation of those ideas.

9. Take some risks in order to make things better. Sometimes you have to bet big to win big.  Sometimes you have to do a controversial thing, or not get permission before you do something that will greatly improve development quality or output.  Increasing developer quality through creating developer tools is very valuable and very visible.  Just make sure if you take a risk that if you accomplish it, it will have high returns for the company.  People who do things without permission that are stupid get fired.  People who do things without permission that are brilliant get praised.

The bottom line is if you’re not selling your self, you’re selling yourself short.

First let me clarify by defining what I mean by “selling yourself”.  I don’t mean “selling out”, I mean marketing yourself, what you’re doing and what your skills are.  Especially to your organization.

I don’t need to sell myself, my code speaks for itself.

Really?  Do you think your pristine code says enough about your skills especially to a non or semi-technical person?  Think back about how many products with fancy UIs beat out excellent products with crappy UIs.  The simple fact of life is that marketing is effective and required.  Imagine creating a really good product and doing no advertising at all because you think that the product speaks for itself.  No one will know about how good it is and no one will buy it.

The only person who can recognize the quality of your code is another developer of equal or greater talent. Now I realize that someone of lesser skill may be able to recognize that your code is “good”, because it is easy for them to read and it looks pretty, but just like a connoisseur of fine wines can distinguish the small differences that really count, it takes a developer of significant talent to recognize excellent code from good or even great code.  Being armed with this knowledge it is easy to see why your code will rarely speak for itself.

As a professional developer you have to consider your audience.  Mostly the people influencing your career in development are not the highly talented developers who can appreciate your code.  In my experience, it is a collective of all developers, project managers, and recruiters.  Because of this audience, you must do more than write good code.  That good code must be seen and the effects of it must also be seen.

It’s wrong to sell myself, it is politics, I don’t do politics, I just write code.

If it’s so wrong to do, how come so many of us do it when we are interviewing for a new job?  It’s like a courtship when they guy is taking the gal out to dinner and buying her flowers and romancing her, but as soon as they get married where did the romance go?  Many of us developers are just like that.  We polish our resumé and go out looking for the job, dressed up in a suit and tie, presenting ourselves as professionals, but then we get the job and we settle in, we turn off the charm.  It’s not wrong to keep the charm turned on, not only is it not wrong, it’s expected.

How about the politics of it?  Well yes, there are politics involved in selling yourself, but thinking  you can be a technical person and completely ignore politics is a sure way to dead-end you career.  Most developers want to reduce the amount of political situations they have to deal with, and I agree with that, but trying to ignore the problem to just make it go away is like trying to make a hungry lion go away by closing your eyes.  Even if you are the best hacker in the world people skills are important.  It is my strong opinion that every professional, developer or not, should read How to Win Friends and Influence People, by Dale Carnegie.

I think there is also a large difference between tasteful marketing of a product or service and offensive or annoying marketing.  Just like the XXX spam mail that shows up in your email unrequested, a developer can market themselves in an unsolicited and annoying way.  On the the other hand…

See part two of the series here: Selling Yourself: How

Jeff Atwood blogged about it here

I think it started from here

Anyway, if you don’t want to click those links and you’re not familiar with it, is a small programming problem designed to screen someone who can actually write code from someone who pretends to write code.  The problem is:

Write a program that prints the numbers from 1 to 100. But for multiples of three print “Fizz” instead of the number and for the multiples of five print “Buzz”. For numbers which are multiples of both three and five print “FizzBuzz”

The point is this is a simple problem.  Just simple enough to be interesting.  I actually got asked to solve this problem this week.  I solved it after bumbling through it way more than I should have (I was trying to think of an elegant solution on the spot.)  But it kept bugging me.  I knew there was a solution that I wanted to give it that would make me happy with feelings of maps, function pointers and linq.  Pretty much 3 of my favorite things.  So I thought about it for a bit and here is the solution I came up with (3rd cut.)

        public static void FizzBuzz()
        {
            Dictionary<Func<int, bool>, Func<int, string>> rules = new Dictionary<Func<int, bool>, Func<int, string>>();
            rules.Add(x => x % 3 == 0, x => "fizz");
            rules.Add(x => x % 5 == 0, x => "buzz");
            rules.Add(x => x % 5 != 0 && x % 3 != 0, x => x.ToString());
            rules.Add(x => true, x => "\n");

            var output = from n in Enumerable.Range(1, 100)
                         from f in rules
                         where f.Key(n)
                         select f.Value(n);

            output.ToList().ForEach(x => Console.Write(x));
        }

That’s my crack at it. If you’re not familiar with all the C# magic going on here, basically it works like this:

You have a map which maps one lambda expression which defines the rule we are checking for to a second lambda expression which is what string to return if that rule matches.
Then we have a LINQ query that does a Cartesian join on the set of numbers from 1 to 100 and the 4 rules.
Finally, we take the output and for each string we write it out.

The 4th rule in the set makes sure that after evaluating the set of rules for each number a newline gets put in.

It is kind of interesting how thinking about different ways to solve a fairly simple problem can help you think about the tools you have in new ways.