Requirements Verification and Validation Activities

Requirement Review Inspection Checklist

 Criteria Matrix:

Verifiable	Y/N	Comprehensible	Y/N	Traceable	Y/N	Adaptable	Y/N	Unique	Y/N
Testable		No Aggregation		System Model		Not Dependency Target of other Reqs		Requirement Unique
Success / Failure Discretization		Precise		Existing Dependencies		No Derivative Requirements
		Consistent				No “how-to”
		Complete

 

Goal: All criteria evaluate to “Y” before release to the System Requirements Specification (SRS).

Inspection Methodology:

_______________________________________________________________

Requirement must be testable:

A requirement will ultimately be verified by running tests against the system and analyzing the results. This method implies that requirement must manifest some behavior that can be verified by testing. Specific values are required for tests and requirements must contain these specific values. Qualitative (non-functional) requirements may be more prone to loose definitions. Qualitative requirements must specify values given by the customer or agreed to by the customer.

Testable Example: "System shall respond to temperature change input within 1 second."

_______________________________________________________________

Requirement must clearly distinguish success from failure:

A given test variation can only have one of two possible results, success or failure. These are discrete values. There cannot be a continuum between success and failure. There should be no partial credit."Almost success", "partial failure", "acceptable, but needs improvement" are all inappropriate test results. A test variation should either pass or fail. If that is difficult or impossible to do, then it is likely that the requirement being tested did not clearly define the difference between what is considered successful and what is cosidered a failure. Requirements must clearly define what is considered successful fulfillment of the requirement (success case). Anything else is failure. The line between success and failure cannot be fuzzy.

Discretized Example: "System shall detect carbon monoxide levels greater than or equal to 50 PPM (parts per million) within 5 feet of the outer edge of the heating unit and report an alert to the user within 2 seconds."
Comment: So, a test that stimulates a condition of 51 PPM carbon monoxide within 5 feet of the heating unit should produce a user alert withing 2 seconds." If no alert is produce, then failure. If the alert is produced at exactly 3 seconds after the precipitating condition, then failure. The success case is clearly specified.

Non-Discretized Example: “System shall maintain 2 millisecond polling intervals when possible.”
Comment: Either a 2 millisecond polling interval is required or it is not.  Should a 2.5 second interval between polling actions be accepted as a success? Only the customer can answer that question. In cases where tolerance ranges are acceptable, the requirement specification should define the tolerance range in its description so that clear thresholds can be established and used to deterministically distinguish successes from failures.

_______________________________________________________________

Requirements must specify a single attribute or behavior:

Requirements must express a single attribute or behavior so that test variations and the requirements they test can have a one-to-one relationship. Aggregate requirements contain a mixed message and are difficult to test appropriately. Aggregate requirements might also imply a coupling between components which would be a design detriment.

Ex: “System shall handle all fatal exceptions and log all errors as they occur.” This requirement is actually an aggregate of two similar, but distinct requirements.  Requirement1: System shall handle all fatal exceptions.

Requirement2: System shall log all errors as they occur.

Microwave oven example of aggregate requirement: "F4.3.2: Each preset button shall have the capability of having a unique cooking time and power setting associated with it."

The above microwave oven example requirement should be split up into two requirements as was done in the previous example.

_______________________________________________________________

Requirements must be precise and have a single, clear meaning:

Requirements must clearly describe what is wanted. Requirements cannot be imprecise or ambiguous. If the requirement is vague, then verificaton of its fulfillment will likely also be vague and hard to prove to the customer. If there is any room for interpretation, two different software engineers are likely to have two different interpretations thus resulting in two different designs and resulting behaviors. This would result in the wrong system being delivered to the customer which is total failure.

Ex: “System shall process transactions in an efficient manner.” This example (just as the previous ones) is contrived to illustrate a point. This requirement is useless as written. What I consider efficient versus what the customer considers efficient may be quite different. Requirements need specific values and descriptive details to be precise. I would suggest the following rewording:

System shall complete processing of a given, single transaction packet in less than or equal to 1.5 milliseconds.

_______________________________________________________________

Requirements must be consistent both with themselves and with other requirements:

A requirement should not contradict itself. It may be impossible to fulfill a contradictory requirement.

Ex: “System shall use the organization’s existing local area network (LAN) intranet, but shall not add to existing network traffic.” This requirement is inconsistent in that it requires the system to be developed to integrate into the company’s existing LAN, but it cannot impact the throughput of that network. Obviously, use of a resource reduces the availability of that resource. A requirement like this must be rejected as an invalid requirement.

Likewise, a requirement should not contradict another requirement. If there are two contradictory requirements, then only one can win. This guarantees unsuccessful fulfillment of at least one of those requirements.

Ex:
Requirement X: "The system shall only use statically-linked libraries."
Requirement Y: "The system shall allow plug-ins to be loaded as necessary."
This contrived example presents an obvious technical problem. Since plug-ins rely on dynamically loading shared objects, they are not possible if the executable is only allowed to link statically to its libraries. Unfortunately, in real life, contradictions are seldom as clear as what is presented in this contrived example.

Requirements should also use consistent terminology. This can be very helpful when search requirements for certain key words during an analysis. Examples of inconsistent terminology follow from the microwave oven example.

Ex:
Requirement F2: "A tone shall sound each time the door is opened or closed."
Requirement F4.2.5: "Within 200 milliseconds after the timer decrements from the value of 00:01 to the value 00:00 a buzzer shall sound."

Both of the above requirements refer to the sound made by the beeper device. They should both either use "a buzzer shall sound" or "a tone shall sound".

Ex:

Requirement F4.3.2: "Each preset button shall have the capability of having a unique cooking time and power setting associated with it. "
Requirement F4.4.6: "The auto defrost function shall require that the start key be pressed when the user is ready to begin the defrosting operation."

Both of the above requirements refer to keypad buttons / keys. They should both either use the term "button" or "key".

_______________________________________________________________

Requirements must be complete:

A requirement must provide enough detail to clearly and precisely convey the need that is being expressed. A requirement needs to be self-contained and able to stand alone.  Requirements should not have a lot of references to other documents as part of their specification. In my opinion, a requirement should not have any references to any other source of information. If the requirement specifies functionality that must comply with some standardized specification, then the requirement should be written in such a way that it encompasses the restrictions, structure, etc. imposed by the specification. Requirements sprinkled with TBDs (To Be Determined) are too embryonic to be of any use and should be rejected as invalid until the customer has a clearer idea of what they want. Requirements deemed to have dependencies, but not noting their dependencies in the specification should be re-analyzed to ensure that all dependencies are noted and captured in the specification. Incomplete requirements may be worse than not have those requirements at all. Incomplete requirements provide fertile ground for scope creep and sometimes missing the point completely. Bad/incomplete information can be worse than no information since it implies customer approval whereas no information encourages additional communication with the customer.

_______________________________________________________________ 

Requirements must be traceable:

A requirement that cannot be traced to the system model used to model the customer’s inputs should be viewed skeptically. Requirements of dubious origin are likely a manifestation of “gold-plating” (adding unnecessary features) and should be pruned from the SRS. Build what the customer wants and only what the customer wants.

_______________________________________________________________ 

Requirement with dependencies must note or trace those dependencies:

A requirement that is separate from the requirement with which it has a dependency relationship should clearly indicate that dependency in its specification. There should also be a dependency matrix that shows all requirements on a grid with a mark denoting dependencies between requirements at their intersection on the grid. Requirements that are known to have dependencies, but have no dependency information in their specification need to be re-analyzed ensuring that all dependencies are captured before being added to the SRS.

_______________________________________________________________ 

Requirements should be independent of other requirements:

A requirement that is depended upon by other requirements may ultimately present a design challenge. Any change to this requirement will likely ripple out to the dependent requirements as well. Not all dependencies are bad; however, a requirement that has a dependency on another requirement may in fact need to be removed as a separate requirement and instead be added as additional detail to the requirement on which it depends. This type of requirements pruning is difficult and subjective. Dependencies may be unavoidable and when they are, the requirement that is depended upon may not be very adaptable to change.

_______________________________________________________________ 

Requirements should have enough detail to avoid the need for many derived requirements:

Similar to the dependency relationship between requirements, when a requirement is the source of many other derived requirements, its changes will likely affect several and perhaps even all of the derived requirements. This situation makes the parent or source requirement less adaptable to any changes. In such a case, it might be advisable to talk to the customer and attempt to get the requirement rewritten in more precise terms. Sometimes, a multitude of derived requirements is simply unavoidable.

_______________________________________________________________ 

Requirements must describe what is wanted, not how to implement it:

A requirement that contains too much "how-to" information is in bad form since requirements should not arbitrarily constrain design options. Given that, customers may insist that a certain type of technology or even architecture be used to implement the system they are purchasing. Any time that a requirement is tied to specific technologies and tool sets, it will be very brittle and not adaptable to change.

Ex: “System shall be implemented by exclusively using Java technologies.” Then the following change arrives in the form of a late requirement: “System shall be deployed as a set of x86 executables.” The customer may not have been technologically savvy and read an article about the impressive attributes of Java and decided that was what they wanted. Later on, they found out that some of their deployment targets would not be allowed to run the Java virtual machine and the system would have to be installed as executables. The brittle requirement has now been completed shattered by a delivery packaging change.