Last week we spoke to a company attempting to integrate test automation into their Agile development process. Confusion abounded when it came to the plethora of tool choices. To add to their uncertainty, they were unsure which team within their group would have responsibility for the selected tool’s implementation and usage. This organization knows they want an open source tool. While new automation tools in this category spring up each year, some of the more popular ones include:
- NUnit
- JUnit
- FitNesse
- Selenium
- Ruby Watir
Our recommendation to the company when evaluating the available automation software, was to consider them falling into one of three categories:
1. white box testing
2. gray box testing
3. black box testing
White Box Testing
White box test tools include NUnit and JUnit. These automation tools play a key role in continuous integration (CI) build processes by validating snippets of code, and its scripts are typically created by developers. In an agile environment, the developers still often write the JUnit or NUnit tests but Zenergy recommends that QA should get involved by reviewing those tests. Once the tests are created and reviewed, a member of the development team, CI team, or QA team will execute them, generally on a consistent basis.
An example of a gray box test tool is FitNesse. This automation tool will test a function of the application. These types of automated scripts are usually written by members of the QA team with help from developers. Most gray box test tools will require development of “hooks” or “fixtures” that must be integrated into the application code. In an Agile environment, QA will create the automated scripts with collaboration from the BA and developer. Members of the QA team, developers, or BAs can execute these types of automated tests.
Black box test automation tools are considered for more “traditional” test automation needs, such as automating manual testing on large enterprise systems. Tool examples are QuickTest Professional, Rational Functional Tester, and TestComplete. They need not be integrated into the application code and work by interacting with the application’s objects. Selenium and Ruby Watir are examples of black box testing tools that test multiple functionalities within the application. The test scripts are typically generated by QA with collaboration from BAs and developers. Members of the QA team, developers, or BAs can run this type of automated tests.
So which automation tool is right for your organization? It depends on the level of testing, test coverage, skill sets of your QA personnel, and the technology under test. However, in an ideal situation, a combination of a white box tool and a gray box or black box tool will significantly increase your test coverage and reduce the risk of defects creeping into production.
Next up:
Are you new to test automation?
Join us for one of our upcoming automation training workshops.
