This is my first in a series of posts about getting to fully automated blackbox testing, or BAT, as I like to call it.
In my last post in the Back to Basics series, I mentioned that I would begin this series with the intent of showing you step by step how to go from no automation to a quality BAT infrastructure.
By the end of this series you should have the information you need to actually get automated testing going in your organization instead of just talking about it.
First stop on this train is hiring an automation lead.
Don’t doom yourself before you start
If you think you are going to create BATs, but you are not going to have at least 1 dedicated resource to doing the job of getting this all going, you are probably bound for failure.
Listen up, because this is the single biggest mistake that most organizations make when trying to get BATs up.
This mistake is thinking that you can get your QA team to get this going and you can part time some developers onto the project, but not having a dedicated resource.
Why is this absolutely absurd and you should expect it to fail? Because, in order to get an infrastructure to support your BATs, you are going to need to basically design and create a testing framework that is tailored to the business domain of your application.
Blah, what? Ok, in simple terms. Someone is going to need to design and build an automation framework.
If you think you are just going to buy some tool and use it to record test scripts, you are wrong. That approach simply is not going to work. I won’t get into why here, but I have written on it before in the past, and if you are interested, let me know and I’ll go much more in depth.
Think about this, do you want your QA team, as good as they may be at QA work, designing and building an automation framework? Give them a few C# or Java classes and have their first real coding work be to design a complex system?
How about a developer who has “real” work and can only devote part time to designing what may end up becoming one of the most important parts of your infrastructure?
It is fine to pull a developer off of the main product and put them in this role until it is up and running, but make no mistake, you need a dedicated resource.
You are going to want to a well designed maintainable automation framework, because the ease of use of that framework will determine how easy or not it is to write automated tests that use that framework.
I recommend getting someone who has done it before
Why? Because creating an automation framework is much different than developing your software and it is much different than writing tests. It really requires a special skill set that is not so easily acquired without the experience of having done this before.
I also think it is fine to use a developer resource, but if you do, you want to try to get a more senior developer to take this role.
Here is a list of some of the skills that I think are important which may also make it much more clear why I am recommending a specialist in this case:
Development of APIs (In the end the automation framework will be an API that is used to write automated tests. The better the API is designed, the easier it will be to write tests against it.)
Design of reusable base components (Much of the framework consists of repeated concepts with different variations for different screens in your application. As a result, the core of the framework will be a basic design that allows for variation without too much repetition… reuse)
Ability to quickly acquire domain knowledge of the software (Someone writing an automation framework needs to really understand how the system under test works, so that they can model the framework after it.)
Pragmatism (Many “accepted” good software designs are actually bad automation framework designs. Static methods can make an automation framework’s API much more simple to use, especially for non-programmers, but would not be a good design choice in your main product. A person who designs an automation framework should be pragmatic enough to “break the rules” in order to make the framework easiest to use and maintain.)
Experience or understanding of QA (It is important to understand the users of the framework, the QA team. Understanding what kind of tests they will likely write or how they will think will go a long way in coming up with a good framework design.)
Communication skills (Good BATs read like English. It is critical to make sure that the person designing the framework facilitates this to make the BATs as useful and understandable as possible.)
HTML / XPath (This specific skill set is very important for web based products. Creating an automation framework built on top of one of the web browser drivers will require intimate knowledge of HTML and probably XPath to be able to access attributes and elements on pages in your application.)
I’ve written a few automation frameworks and contributed to the design of several others, and I can tell you that my version 6.0 of an automation framework was much much better than version 1.0.
There are a huge number of patterns that emerge that are unique to this problem domain. So, I do believe having experience in this area specially is a huge benefit.
What does this automation person do?
We’ll get to more specific in my later posts in this series, but the basic responsibility should be:
- Figure out the technology and tools to be used
- Create the initial automation framework
- Create smoke tests for the application under test
- Get those smoke tests into some kind of automated build
- Train developers and QA in the use of the framework
- Drive the adoption of the framework and automation for all features
- Set up rules for consistency in writing automated tests
- Design of hierarchy and structure of automated tests
You want your automation person to be responsible for pushing and driving forward the goal of getting to the point of being able to automate each feature being created with minimal effort.
This is one of those jobs that can actually be “done.” What I mean by that is you may have the automation lead end up creating the framework, developing it enough to push it out to your development teams and train them to write automated tests along with QA, and you really don’t need an automation lead anymore.
This is great though, because now you have a developer that really knows your product and is a good bridge to QA, or you can keep them improving the framework and overseeing it if you want to go that route.
Bottom line is:
- Make sure you get a dedicated resource
- Make sure that resource can write code and design a framework
- Make sure they train everyone else, one person cannot automate your entire application