How we integrated our distributed testing framework with Selenium for automating our Web UI testing
GigaSpaces XAP – as a distributed application platform and data grid – has some unique and interesting testing requirements. I’d like to explain how we use Selenium to test our upcoming web-based administration console in the grid.
Introduction on SGTest testing environment
GigaSpaces Web-UI testing is part of our Service Grid Testing (“SGTest”) project, a distributed testing environment that enables us to emulate how a user would use GigaSpaces features, and allows us to run multiple tests, on multiple machines, and in many different realistic real world scenarios.
To manipulate the grid components – the containers, managers, lookup services, agents, and the rest – we use our Admin API. The Web-UI tests run mainly in our setup, consisting of 1 GSM, 1 GSA, 2 GSCs, and 1 LUS. The tests are configured and managed by the Java TestNG testing framework.
So we have three primary components in our administration UI testing system: the test environment (SGTest), the testing framework (TestNG), and Selenium.
What is Selenium?
Selenium is a testing tool, designed especially for testing web applications. It can interact with an application almost as well as an actual human being can. (In fact, since it doesn’t make any of the silly mistakes humans make, maybe it can interact better.)
UI Testing can be brutal, consisting of long scripts clicking on events in order, hoping no steps are missed. Selenium can help, by programmatically interacting with the application – almost all being done with code of our desired programming language – in this case, Java.
How does it work?
Selenium tests can be executed in two ways.
One is just to run the test case from the Selenium IDE, which is a Firefox add-on seen in the below screen shot on the left hand side. While this is one way to go, in our case it’s not really (at all) what we were looking for. We wanted to be able to integrate the tests into Eclipse, so we can use the features SGTest and TestNG have to offer us, such as the ability to use the Admin API as well as TestNG annotations.
Enter the Selenium RC server. Its job is manage browser instances, and intercept user interface actions from a client, and then execute them on the web application.
Creating a test case
So now we have the pieces in place: Eclipse is our local development application, we’re using Selenium RC to handle the execution of user interaction with our web application, TestNG is the framework we use to start our tests, and SGTest is the environment we’re testing against.
How do we actually create a test? By running an interaction manually, as one option.
The Selenium IDE is able to record interaction with an application. This recording generates code that emulates the recorded interaction. For example, a user clicking on a button called "Submit" will generate the code selenium.click("//button[@id=submit]");, which makes sense if you consider what the code is saying (“act like a user has clicked on a node in the browser DOM with a name of ‘button’ and an id of ‘submit’”).
So, in general writing a test consists of three steps:
- Recording the desired test scenario using the Selenium IDE Firefox plug-in.
This process is essentially executing the test manually; which is done only once, in Firefox.
- Copying the generated code to a new Java class.
- Manipulating the generated code as we wish from eclipse.
Creating the infrastructure
Our testing infrastructure has 3 parts:
Servers
To view our Web-UI and interact with it, we need to have the web UI actually running (as one might expect.) Therefore, we need to run the gs-webui script manually in the system console. This launches our Web-UI server, which enables the use of the UI.
This means that in order to run tests, we need this server running.
As mentioned before, the Selenium RC server needs to be up as well.
These servers should be launched once, before any test has been executed. After all tests are finished, we want to shut them down.
Libraries
Selenium provides us with a set of jars that we add to our classpath. This is so that Eclipse will recognize Selenium commands. These provide the language-level binding for Selenium RC’s user interactions.
XML
We create two xml files. One called testng.xml; this xml is just our configuration of the TestNG test suite. It defines the suite name, and classes belonging to the suite. We choose to include all of our classes containing tests in the suite, although this is not mandatory.
The second xml file is call test-param.xml; sometimes we want to run the same test a few times, each time with different parameters. This xml file contains all test permutations. By using a special TestNG annotation we can inject these parameters into our test methods.
So, how do we put all of this together? The Selenium jars have a certain inheritance hierarchy; the base class is called SeleniumTestNGHelper. Its main purpose is to provide us with a Selenium object instance, and to launch the browser of our choice where the tests will run. All Selenium commands are visible only via the Selenium instance, so this is essential.
What this means is that tests have to extend this class. But we have a little tweak: instead of having our tests extend SeleniumTestNGHelper, we wrote an abstract class called AbstractSeleniumTest, which handles the issues of making sure the servers are correctly configured and running. All tests now extend this class, which in turn extends SeleniumTestNGHelper.
What is tested? What is not tested?
Our Web-UI is developed with GWT. Selenium recognizes these GWT elements and emulates a user interaction with the application. We test all possible features that appear in the UI (i.e. deployment, statistic viewing, etc.)
The only apparent shortcoming of Selenium is that it is unable to capture OS components, like the file dialog for saving/loading files. Tests of this nature are done manually.
Therefore, we are able to test every interaction with the system that isn’t relying on the OS itself, as file dialogs do. The OS-specific features get tested through normal methods.
Example
Let’s take a look at an example test, step by step. This test undeploys an existing processing unit and asserts that its status has changed to undeployed. It then asserts that it is no longer visible in the UI.
Every test extends AbstractSeleniumTest, as we have explained:
Next we write a method for starting our desired Grid services. In this case, we use the default configuration (one manager, two containers), and also deploy a processing unit using the Admin API:
Now we write our test method. This method is the one generated by the Selenium IDE, as it watches us interact manually with the UI:
All Selenium commands have a string as an argument. This argument is called a Locator, and as the name implies, it is used to locate elements in the UI. There are various locating techniques we can use: Xpath, CSS, DOM, etc.; these help us ensure that in case the UI changes, the impact on the test is minimal.
That’s pretty much it. Note how the test uses all 3 participants I mentioned earlier: the SGTest enables use of the Admin API; the TestNG annotations help us control the flow of the test; of course, Selenium does the actual work.
Summary
I am sure you all know the importance of automating tests for your application, whenever such automation is possible; people are much more prone to mistakes than computers are. This cannot be stressed enough.
As you can see, it is fairly easy to write tests with Selenium. The syntax and logic is highly intuitive and I have found working with this tool very satisfying.
There are of course many more capabilities to Selenium not specified here, but hopefully I have given you a pretty good idea on how GigaSpaces uses Selenium to automate our Web-UI testing as much as possible.
Cheers,
Eli
