Category Archives: Watij

Boise Code Camp 2010 Presentations

Went to Boise Code Camp this weekend and it was great!

There were some really good sessions and good discussions on Agile, Scrum and domain specific languages.

I think events like these are important because they help us to zoom out a little and see what kinds of things other people are doing in our field and what companies around us are interested in high quality software.

Code camps are also a great place to get some experience doing presentations and share what you are working on.  Networking opportunities abound.

I presented two talks this weekend.  I am posting the links to those here.

The first presentation I did was on Pair Programming.

You can find the presentation on Prezi here.

We also did a live pair programming session, which I thought went pretty well.  We ran into some of the issues that often come up with pair programming, like being afraid to tell someone you don’t like their method name.

The second presentation I did was on Creating an Internal DSL for Automated Functional Testing.

You can find the presentation on Slideshare here.

We went over how to go about creating a good automation framework which fills in the gaps between the business language and technical language.

Thanks to everyone who attended.

Automated UI Testing Framework… A Real Example

Michael Feathers recently blogged a very interesting topic he titled “UI Test Automation Tools are Snake Oil”.  This seems to have stirred up again the very controversial topic of automated UI testing.  I had previously posted about the pro’s and con’s of using a recording tool versus creating a framework.  Now I wanted to expand a little more on using a framework by giving a concrete example of a framework I have written and am using to actually write good automated UI tests that are not fragile.

The project

The project I am working on is mostly written in Java.  It is a case management system which workers use to enter applicants’ information and then run their eligibility to qualify them for benefits.  The basic use of the system is to create a new case, create people on the case with their information: income, relationships etc, and then run the eligibility and save the results.  One other important piece here I want to mention before I get flamed about technology choices is that the product was a legacy product and only supported on IE.

The tools

Based on most of the developers being versed in Java and the system requiring IE, I chose to use Watij for the framework.  Watij is basically a framework built on top of IE that directly interacts with the browser through the DOM through the COM interface.  When you are testing in Watij you are actually interacting with the browser, but since you are going through the com layer instead of clicking directly through the screens you are able to hide the browser window and run tests with the machine locked.  I also used JUnit as the test driver to execute the tests since it is easy to run from eclipse and can easily be run from an ant build to report the results and run on the build server.

The framework

The basic idea behind this framework is that I wanted to make it so a non programmer could write the tests and maintain the tests without having to know much about Java or Watij.  Even though Watij happens to be being used behind the scenes, I wanted the person writing the test scripts to only need to have domain knowledge of the application.  With that in mind, here is a picture of the stack.

 Automated UI Testing Framework... A Real Example

You can see in the diagram that I show a technology and how you interact with it.  IE uses HTML and CSS, Watij is built on top of IE and lets you work against the DOM.  My application framework is build on top of Watij and lets you work with domain specific concepts such as a specific screen, work flows and navigation.  The test scripts are written against the domain specific application framework.

Using this type of a framework, most tests can be written in as few a 5 lines of code.  Here is an example of a test:

@Before
     public void SetUp()
     {
          this.caseNumber = CaseCreator.createCase( IBESDate.currentMonth(),
                                                    false,
                                                    false,
                                                    People.ADULT_MALE_1.onPrograms( Program.MEDICAID ) );
     }

     @Test
     public void taskCanBeSelected()
     {
          Tasks.addTask( TaskType.RECEIVED_458, IBESDate.monthsInFuture( 3 ) );
          WorkQueue.search( TaskType.RECEIVED_458,
                            IBESDate.monthsInFuture( 3 ),
                            IBESDate.monthsInFuture( 3 ) );

          Assert.assertTrue( WorkQueue.searchResultsHasCase( this.caseNumber ) );

     }

Now this isn’t going to make much sense to most people who read it, and I actually consider that a good thing.  If you looked at the screens in the application it would make much more sense.  Someone writing these tests only needs to know about the screens and they can figure out how to do something.  There is actually a large amount of things going on in the background when you run this test.  It will create a complete case, add the person to the case, then click through a task workflow to add a specific task, then it will go through another workflow to search by the task to see that it was added.  There are probably about 20-25 different screen that this test ends up going through with these very few and simple lines of code.

The secret behind these kinds of a simple UI automation tests is abstraction.  The framework is designed to essentially be a domain specific language.  By abstracting away anything that has to do with the browser, html and even the dom, I am able to present a simple and stable interface for the tester to do exactly what they want and not know about all the other technology going on behind the wheel.  Think about how complex a motor vehicle is, yet how simple the interface to the user is.  I see many automation strategies that require the person writing the test scripts to know about all nuances of browsers and html.

What about change?

Most automation strategies break down when encountering UI changes.  I welcome UI changes.  Go ahead and change the UI, it won’t break all my tests, it will break one specific place in the framework that knows about what kind of things are on a particular screen.  With this framework it is very easy to fix a broken UI in one place and all the tests will work again.  Contrast this to many of the automation scenarios you have seen where a single renaming of a button requires 50 tests to be reworked.  All of the Watij interaction is in one namespace called “UI”.  All of the screens use UI to interact with Watij, so that I could even swap out Watij for another browser automation framework.

Back to the snake oil

I do have to say I agree with many of the points Michael made in his post, but I have seen them overcome.  A tool is never going to be a magic bullet.  Yet at the same time we should not run away from UI automation.   We just need to correctly apply the same principles we apply to create good abstractions in our source code to our automation code.

Automated Functional Tests: Record or Program?

Recently, I’ve been devoting some time to building out a framework using Watij with the goal to be able to easily write automated functional tests for backlog items for the application I am working on.  There are many factors in the technology choice, but one of the most prominent of questions about doing automation seems to be around whether or not to use a tool that records test scripts vs using a WatiX or similar library and rolling your own framework.  Let me put a little caveat here, since I know this will come up.  If you use a recoding tool and then you use its internal language to build a framework that is actually reusable and makes it so you don’t have to re-record every time you create a new test, I would count that as programming vs recording.  When I say recording, I mean mostly just recording and plugging in variables.

Let’s looks at the pro’s and con’s:

Recording

+ Very fast to get started, you can have a working test right from the get go.

+ Anyone can do it.  Just requires a little training in the tool, but not much technical knowledge needed.

+ Easy execution of tests, usually integration with test case management systems, defect systems, etc.

- Speed of test creation doesn’t really increase as test library increases.

- Lots of pointing and clicking to create the tests each time.

- Fragile: if a screen changes it can break many tests which each have to be changed.

- Expensive, most of the recoding tools are very expensive.

- Heavy installation, most of the tools require thick clients to use, installed on any machine recording or running them.

Programming

+ As your library increases in size, test case creation becomes very rapid.

+ Developers are more likely to participate since it is a programming type activity.

+ Can abstract screen changes from functions it order to protect against change.  (If one screen changes you are more likely to only need to update the framework in one spot, not change every single test.)

+ Low cost, as in free, solution.  (Besides development cost)

+ Can run on any machine that has Ruby, .NET Framework, or Java depending on framework used.

– Takes at least some development expertise to build the initial framework.

– Requires more training to understand how to use.

– Initial creation of test cases takes a long time.

Conclusion:

The patterns of pro’s and con’s here closely resembles the pattern of pro’s and con’s for doing unit testing or test driven development.  Higher cost initially, but over time large amount of savings.  Programming seems to be a clear winner over recording, in my opinion.

Next time I’ll take a look at how to implement a good framework for automated functional testing using a tool like Watij, Watin, or Watir.

Updated, here is the post that looks at the action implementation of the framework.

Difficult Watij Problem That Wasn’t Really A Watij Problem

Don’t you hate it when you spend a large amount of time dealing with a problem only to find out what you thought was the issue really wasn’t the issue after all? I spent a significant amount of time today trying to find out why I couldn’t access a link that I can see on a web page and in the DOM when using the IE DevToolBar. I thought it was a problem with the automated testing framework I use, Watij. But, it turns out what I thought was a Watij problem really wasn’t.

I was trying to write an automated acceptance test to verify that a birth date could not be made null.

Pop quiz, what’s wrong with this HTML?

<!-- This is a comment --!>

It turns out that IE can recognize that “–!>” is intended to be the end of a comment even though it should be:

<!-- This is a comment -->

Unfortunately, Watij is not as forgiving and does not recognize the intention behind this mistake.  In fact, Watij treats the comment as not ending, and the rest of the page as a comment.

So, the lesson behind this, is that if you are using a WatiX framework and can’t see a link or image on a page, be sure to check and make sure you have well formed comments. Better yet, anytime  you come across something similar, make sure all your html is well formed.