Roles and Activities >
Developer Role Set >
Test Designer >
Define Testability Elements
Roles and Activities >
Developer Role Set >
Test Designer >
Define Testability Elements
Activity:
| ||||||||||||||||||||||||||||||||||||
| Workflow Details: |
| Purpose: | To gain an understanding of how each required test mechanism will need to support the target test items. |
Taking each test mechanism in turn, review the target test items. Identify what target items will require the support of each mechanism. Remember that the target items include the software being developed, third-party software elements, operating systems and hardware. Test mechanisms may need to be implemented to support the testing of any of these items.
| Purpose: | To gain an understanding of the dynamic and run-time aspects of the system. |
Using the available software requirements and design information, identify the dynamic elements and events of the system. Using the use-case, design, implementation and deployment models, you can identify relevant items such as control classes, processes, threads and events. Places to begin your research include classes stereotyped as <<control>>, use-case realizations, and elements described in the process architectural view or the implementation model stereotyped as <<process>> or <<thread>>.
In relation to the constraints imposed by the test environment, define the physical requirements
| Purpose: | To gain an understanding of the responsibilities of the system as a service provider, and the dependencies of the system as a client . |
Another useful group of elements to examine are the Interfaces of the system, most importantly those that relate to actors external to the boundaries of the system. Using the Design and Implementation Models, look for elements defined with the stereotype <<interface>>. Also examine the models for the existence of classes stereotyped as <<boundary>>.
As a tester, it is useful to explore past these system boundaries to gain an understanding of the expectations of the related systems, both client and service providers. This will give you a more thorough understanding of what is needed both in terms of validation of the interfaces and in terms of the test infrastructure required to test and possibly simulate these interfaces.
| Purpose: | To identify the essential elements of the test effort that will enable the required testing to be performed. |
For an iterative test effort to be successful, it is important to identify and maintain an appropriate infrastructure. Without an infrastructure to help maintain it, the test effort can quickly become unmaintainable and unusable. While more obviously relevant to the automated test effort, test infrastructure is also an important concern for the manual test effort.
Consider the dynamic elements and events in the system; what dependencies will
these place on the implementation of individual tests? Look for opportunities
to uncouple the dependencies between individual tests and manage them through
common points of control that provide a layer of indirection. Common areas to
explore for dependencies include test navigation, test data use and system state
changes.
Using the information you have gathered, consider what requirements will govern
the test infrastructure, and what facilities it will need to provide to enable
a successful test approach.
Some tests have a common structure to the scenario or procedure followed when they are executed, but the same procedure needs to be conducted many times against different test target items. In the case of test automation, it can be useful to create common test scripts or utility functions that can be reused in many different contexts to undertake these common test scenarios in an efficient way. This provides a central point of modification if the test scenario needs to be altered. Examples include conducting standard boundary tests on appropriate classes of interface elements, and validating UI elements for adherence to UI design standards.
When tests are to be conducted in a given test environment configuration, there is the potential for conflicts in the test data values that are used. This problem is compounded when the environment is shared by multiple test team members. Consider using a data-driven approach that uncouples test data values from the test scripts that use them, and provide a central point of collection and modification of the test data. This provides two key benefits; it gives visibility of the test data to all test team members, allowing them to avoid potential conflicts in test data use, and it provides a central point of modification for the test data when it needs to be updated.
Most tests require the system to be in a specific given state before they are executed, and should return the system to a specific known state when they complete. Common dependencies involve security rights (function or data), dynamic or context sensitive data (e.g. system dates, order numbers, user id preferences etc.), data expiry cycles (e.g. security passwords, product expiry etc.). Some tests are highly dependent on each other; for example, one test may create a unique order number and a subsequent test may need to dispatch the same order number.
A common solution is to use test suites to sequence dependent tests in the correct system state order. The test suites can then be coupled with appropriate system recovery and set up utilities. For automated test efforts, some solutions may involve using centralized storage of dynamic system data and the use of variables within the test scripts that reference the centralized information.
Tests sometimes need to calculate or derive appropriate data values from one or more aspects of the runtime system state. This applies to test data values for both input and expected results. Consider developing utilities that calculate the derived data values, simplifying test execution and eliminating potential inaccuracies introduced through human error. Where possible, develop these utilities so that the can be utilized by both manual or automated test efforts.
For test automation, you should consider isolating common navigation sequences and implementing them using centralized utility functions or test scripts. These common navigation sequences can then be reused in many places, providing a central point of modification if the navigation subsequently changes. These common navigation aids simply navigate the application from one point to another; they typically don't perform any tests themselves other than to verify their start and end states.
| Purpose: | To identify the needs of the test discipline that will place potential constraints on the software engineering process, the software architecture and the corresponding design and implementation. |
Especially where test automation is concerned, it's likely that the test implementation
and assessment needs that will place some constraints on both the way the development
team enact the software engineering process, and on the architecture and design
of the software. It's important that the software development team are not unduly
hampered in their core development work and that the test team have the ability
to perform the necessary testing. See Activity:
Obtain Testability Commitment for information about presenting the needs
of the test team to the development team and finding workable solutions that
satisfy the needs of all disciplines.
Using the information you have gathered, consider
what requirements the test effort will place on the development effort.
Consider the interfaces identified; are there additional requirements the test effort will need included in the software design and subsequently exposed in the implementation? In some cases, additional interfaces will be required specifically to support the test effort, or existing interfaces will require additional operating modes or modified message signatures (changes to input and return parameters).
In relation to the target deployment environments (as captured in the test environment configurations) and the development schedule itself, identify the constraints and dependencies placed on the test effort. These dependencies may necessitate the provision of stubs to simulate elements of the environment that will not be available or are too resource prohibitive to establish for testing purposes, or to provide the opportunity the early testing of components of the partially completed system.
Some tests are potentially valuable but prohibitively expensive to implement as true black-box tests. Furthermore, in high-reliability environments it is important to be able to test for and isolate faults as quickly as possible to enable fast resolution. In these cases, it can be useful to build tests directly into the executable software itself.
There are different approaches that can be taken to achieve this; two of the most common include built-in self tests where the software uses redundant processing cycles to perform self-integrity tests, and diagnostic routines that can be performed when the software is sent a diagnostic event message, or when the system is configured to run with diagnostic routines enabled.
Some of the design and implementation choices of the development team will either enable or inhibit the test effort. While some of these choices are unavoidably necessary, there are many smaller decisions—especially in the area of implementation—that have minimal impact on the development team but significant impact on the test team.
Areas to consider include: Use of standard, recognized communication protocols; Use of UI implementation components that can be recognized by test automation tools; Adhering to UI design rules including the naming of UI elements; Consistent use of UI navigation conventions.
| Purpose: | To specify the requirements for the software functions needed to support the implementation and execution of tests. |
Using the previous work performed on the activity, define the test-specific
requirements and constraints that should be considered in the software design
and implementation.
It is important to clearly explain to the development team the reasons why test-specific
features are required to be to built into the software. Key reasons will typically
fall into one of the following areas:
Test-specific features built into the software need to strike a balance between the value of a built-in test feature and the effort necessary to implement and test it. Examples of built-in test features include producing audit logs, self-diagnostic functions and interfaces to interrogate the value of internal variables.
Another common use of test specific functionality is during integration work where there is the need to provide stubs for components or subsystems that are not yet implemented or incorporated. There are two main implementation styles used for stubs:
This second style of stub also provides a powerful means of isolating components or groups of components from the rest of the system, thus providing flexibility in the implementation and execution of tests. As with the earlier comment about test-specific features, a balance between the value of a complex stub and the effort necessary to implement and test the stub needs to be considered. Use this second style prudently for two reasons; first, it takes more resources to implement, and second; it is easier to overlook the existence of the stub and forget to subsequently remove it.
Record your findings in terms of test-specific requirements on the design and implementation models directly, or using one or more test interface specifications.
| Purpose: | To specify the requirements for the test infrastructure needed to support the implementation and execution of tests. |
Using the previous work performed on the activity, define the test infrastructure
that is required to support test implementation and execution.
Remember that you are defining the implementation features of the infrastructure;
The main objective is to define the various parts of the solution that will
implement that infrastructure.
Key requirements or features of the test automation infrastructure include:
Record your decisions as definitions in the implementation sections of the Test Automation Architecture, process guidance in one or more Test Guidelines or as Test Scripts, Test Suites, or test library utility routines. See Artifact: Test Automation Architecture for further suggestions.
Key requirements or features of the manual test infrastructure include:
Record your decisions as process guidance in one or more Artifact: Test Guidelines.
| Purpose: | To verify that the activity has been completed appropriately and that the resulting artifacts are acceptable. |
Now that you have completed the work, it is beneficial to verify that the work was of sufficient value, and that you did not simply consume vast quantities of paper. You should evaluate whether your work is of appropriate quality, and that it is complete enough to be useful to those team members who will make subsequent use of it as input to their work. Where possible, use the checklists provided in RUP to verify that quality and completeness are "good enough".
Have the people performing the downstream activities that rely on your work as input take part in reviewing your interim work. Do this while you still have time available to take action to address their concerns. You should also evaluate your work against the key input artifacts to make sure you have represented them accurately and sufficiently. It may be useful to have the author of the input artifact review your work on this basis.
Try to remember that that RUP is an iterative process and that in many cases artifacts evolve over time. As such, it is not usually necessary—and is often counterproductive—to fully-form an artifact that will only be partially used or will not be used at all in immediately subsequent work. This is because there is a high probability that the situation surrounding the artifact will change—and the assumptions made when the artifact was created proven incorrect—before the artifact is used, resulting in wasted effort and costly rework. Also avoid the trap of spending too many cycles on presentation to the detriment of content value. In project environments where presentation has importance and economic value as a project deliverable, you might want to consider using an administrative resource to perform presentation tasks.
|
Rational Unified
Process
|
