Guidelines: Test
Ideas for Statechart and Flow Diagrams
Topics
This guideline shows how to identify test
ideas from statecharts and other design structures that consist mainly of
nodes connected by arcs and that show something of the possible control flows
of a program. The main goal of this testing is to traverse every arc in some
test. If you've never exercised an arc, why do you think it will work when a
customer does?
Consider this statechart:
Fig1: HVAC Statechart
Here's a first list of test ideas:
- Idle state receives Too Hot event
- Idle state receives Too Cool event
- Cooling/Startup state receives Compressor Running event
- Cooling/Ready state receives Fan Running event
- Cooling/Running state receives OK event
- Cooling/Running state receives Failure event
- Failure state receives Failure Cleared event
- Heating state receives OK event
- Heating state receives Failure event
These test ideas could all be exercised in a single test, or you could create
several tests that each exercise a few. As with all test design, strive for
a balance between the ease of implementation of many simple tests and the additional
defect-finding power of complex tests. (See "test
design using the list" in the Concept:
Test Ideas List page.) If you have use case scenarios that describe certain
paths through the statechart, you should favor tests that take those paths.
In any case, the tests should check that all actions required by the statechart
actually take place. For example, is the alarm started on entry to the Failure
state, then stopped upon exit?
The test should also check that the transition leads to the correct next state.
That can be a difficult problem when the states are invisible from the outside.
The only way to detect an incorrect state is to inject some sequence of events
that leads to incorrect output. More precisely, you would need to construct
a follow-on sequence of events whose externally-visible results for the correct
state differ from those that the same sequence would provoke from each possible
incorrect state.
In the example above, how would you know that the Failure Cleared event in
the Failure state correctly led to the Idle state, instead of staying in the
Failure state? You might trust that the stopping of the Alarm meant that transition
had been made, but it might be better to check by lowering the temperature enough
to make the heater start or raising it enough to turn on cooling. If something
happens, you're more confident that the transition was correct. If nothing happens,
it's likely the device stayed in the Failure state.
At the very least, determining whether the resulting state is correct complicates
test design. It is often better to make the state machine explicit and make
its states visible to the tests.
Other statechart constructs 
Statecharts consist of more than arcs and arrows. Here is a list of statechart
constructs and the effect they have on the test idea list.
Event actions, entry actions, and exit actions
These do not generate test ideas per se. Rather, the tests should check that
the actions behave as specified. If the actions represent substantial programs,
those programs must be tested. The test ideas for the programs might be combined
with test ideas from the statechart, but it's probably more manageable to separate
them. Make the decision based on the effort involved and on your suspicion that
there might be interactions between events. That is, if a particular action
on one arc cannot possibly share data with an action on another arc, there is
no reason to exercise the two actions in the same test (as you would if they
were part of the same path through a statechart test).
Guard conditions
Guard conditions are boolean expressions. The test ideas for guard conditions
are derived as described in Guideline:
Test Ideas for Booleans and Boundaries.
In the example above, the Too Cool transition from the Idle state is guarded
with [restart time >= 5 mins]. That leads to two separate test ideas:
- Idle state receives Too Cool event when restart time is
five minutes (transition taken)
- Idle state receives Too Cool event when restart time is
just less than five minutes (transition blocked)
In both cases, any test that uses the test idea should check that the correct
state is reached.
Internal transitions
An internal transition adds the same sort of ideas to a test idea list as an
external transition does. It's merely that the next state is the same as the
original state. It would be prudent to set up the test such that the state's
entry and exit actions would cause an observable effect if they were incorrectly
triggered.
Nested states
When constructing tests, set them up such that entry and exit events of the
composite state have observable effects. You want to notice if they're skipped.
Concurrent substates
Testing of concurrency falls outside of the scope of developer testing.
Deferred events
If you suspect an event might be handled differently depending on whether it
was deferred and queued rather than generated while the program was actually
in the receiving state, you might test those two cases.
If the event in the receiving state has a guard condition, consider the ramifications
of changes to the condition's variables between the time the event is generated
and the time it is received.
If more than one state can handle a deferred event, consider testing deferral
to each of the possible receiving states. Perhaps the implementation assumes
that the "obvious" state will handle the event.
History states
Here is an example of a history state:
Fig2: History State Example
The transition into the history state represents three real transitions, and
thus three test ideas:
- BackupUp event in Command state leads to Collecting state
- BackupUp event in Command state leads to Copying state
- BackupUp event in Command state leads to CleaningUp state
Chain states
Chain states do not seem to have any implications for test design, except that
they introduce more actions that need to be checked.
The preceding discussion focuses on checking whether the implementation matches
the design. But the design might also be wrong. While examining the design to
find test ideas, also check for two types of problems:
Missing events. The statechart shows a state's response to events that
the designer anticipated could arrive in that state. It's not unknown for
designers to overlook events. For example, in this statechart (repeated from
the top of the page), perhaps the designer forgot that a failure can occur in
the Ready substate of Cooling, not just when the fan is Running.
Fig3: HVAC Statechart
For this reason, it's wise to ask, for each state, whether any of the events
that apply to other states might apply to this one. If you discover that one
does, correct your design.
Incomplete or missing guard conditions. Similarly, perhaps guard conditions
on one transition will suggest guard conditions on others. For example, the
above statechart takes care not to restart the heater too often, but there is
no such restriction on the cooling system. Should there be?
It is also possible that variables used on one guard condition will suggest
that other guard conditions are too simple.
Testing each arc in a graph is by no means complete testing. For example, suppose
the start state initializes a variable to 0, state Setter sets it to 5, and
state Divider divides it into 100 (100/variable). If there's a path from the
start state to Divider that does not pass through Setter, you have a divide-by-zero
exception. If the statechart has many states, simply exercising each arc might
miss that path.
Except for very simple statecharts, testing every path is infeasible. In practice,
tests that are complex and correspond to use case scenarios are often sufficient.
If you desire stronger tests, consider requiring a path from each state where
a datum is given a value to each state that uses it.
Copyright
© 1987 - 2001 Rational Software Corporation
| 
Artifacts >
Test Artifact Set >
Test-Ideas List >
Guidelines >
