Concepts:
Test Automation and Tools
Test automation tools are increasingly being brought to the market to
automate the activities in test. A number of automation tools exist, and it is
unlikely that a single
tool is capable of automating all test activities. Most tools focus on a
specific activity or group of activities, and some only
address one aspect of an activity.
When evaluating different tools for test automation, it is important to be
aware of the type of tool you are evaluating, the limitations of the tool, and what
activities the tool addresses and automates. Test tools are often evaluated and
acquired based on the following categories:
Test tools may be categorized by the function they perform. Typical function
designations for tools include:
- Data acquisition tools that acquire data to be used in
the test activities. The data may be acquired through conversion,
extraction, transformation, or capture of existing data, or through the
generation from use cases or supplemental specifications
- Static measurement tools that analyze information
contained in the design model(s), source code, or other fixed sources. The
analysis yields information on the logic flow, data flow, or quality
metrics, such as complexity, maintainability, or lines of code.
- Dynamic measurement tools that perform an analysis during
the execution of the code. The measurements include the run-time operation
of the code, such as memory error detection and performance.
- Simulators or Drivers that perform
activities that are not, or could not be available for testing purposes, for
reasons of timing, expense, or safety.
- Test management tools that assist in the planning,
design, implementation, execution, evaluation, and management of the test
activities or artifacts.
Test tools are often characterized as either white-box or black-box, based
upon the manner in which the tool is used, or the technology / knowledge needed
to use the tools.
- White-box tools rely upon knowledge of the code, design
model(s), or other source material to implement and execute the tests.
- Black-box tools rely only upon the use cases or
functional description of the target-of-test. Where as white-box tools have
knowledge of how the target-of-test processes the request, black-box tools
rely upon the input and output conditions to evaluate the test.
In addition to the broad classifications of tools presented above, tools may
also be classified by specialization.
- Record/Playback tools combine data acquisition with
dynamic measurement. Test data is acquired during the recording of events
(during test implementation). Later, during test execution, the data is used
to "playback" the test script which is used to evaluate the
execution of the target-of-test.
- Quality metric tools are static measurement tools that
perform a static analysis of the design model(s) or source code to establish
a set of parameters describing the target-of-test's quality. The parameters
may indicate reliability, complexity, maintainability, or other measures of
quality.
- Coverage monitor tools indicate the completeness of
testing by identifying how much of the target-of-test was covered, in some
dimension, during testing. Typical classes of coverage include,
requirements-based (use cases), logic branch or node (code-based), data
state, and function points.
- Test case generators automate the generation of test
data. Test case generators use either a formal specification of the
target-of-test's data inputs or the design model(s) / source code to produce
test data that tests the nominal inputs, error inputs, and limit and
boundary cases.
- Comparator tools compare test results with reference
results and identify differences. Comparators differ in their specificity to
a particular data formats. For example, comparators may be pixel-based (to
compare bitmap images) or object-based (to compare object properties or
data).
- Data extractors provide inputs for test cases from
existing sources. Sources include, databases, data streams in a
communication system, reports, or design model(s) / source code.
Copyright
© 1987 - 2001 Rational Software Corporation
| |
|
Disciplines >
Test >
Concepts >
