The Taxonomy-Based Questionnaire (TBQ) was developed by the Software Engineering Institute(SEI) and first documented the technical report Taxonomy Based Risk Identification [Carr 93]. TBQ is a set of questions organized according to the taxonomy, or division into ordered groups or categories, of project development for the purpose of identifying risks by interviewing a group of one or more individuals (TBQ Interviews).

For this course you shall use the following as a definition for system/software engineering risk management. The definition for Taxonomy-Based Questionnaire (TBQ) is: A questionnaire organized according to the taxonomy of system and software development for the purpose of identifying risks by interviewing a group of one or more individuals.

The taxonomy partitions either a system or software development into components. The Figure SEI Taxonomy Structure is an example of how to partition these relationships.

The software development risk taxonomy has three classes:

1. Product engineering: what you are trying to build
2. Development environment: how you are building it
3. Program constraints: what you can’t control

Under each of the three classes is a series of Elements (requirements) associated with its class.

1. Product engineering

Under each Element are the associated Attributes such as:

• Requirements
• • Stability
  • .....
• Scale

This tree thus represents the orderly classification of system or software development according to their presumed natural relationships.

Example TBQ Questions

This next table is a sample of some TBQ questions. Questions are listed under the attribute. For each attribute there is at least one question.

Hyperlink to Taxonomy_Carr_93

There are three types of questions:

1. Question with cues: (Look at question [22] in the table.) The intent is to ask the question and if the participants have trouble answering then you’d provide some cues to see if they trigger any thoughts.
2. Questions with follow-up question(s): (Look at question [23] in the table.) The intent is to ask the question and based on the response (yes or no) you ask the appropriate follow-up questions.
3. Question only: These questions have no cues or explicit follow-up questions.

The questions are meant to stimulate conversation about possible risks and are not necessarily worded to indicate whether a risk exists or not. Project personnel will ultimately decide whether or not a risk statement should be captured.

For example: Are there any problems with performance? An answer of yes, we have throughput problems doesn’t automatically mean it’s a risk. Natural follow-up questions might be:

Goal/Question/Metric Paradigm GQM

Goal/Question/Metric (GQM) is a paradigm for stating goals and deriving from them questions that need to be answered. These sets of questions provide a specification for the data needed to identify and track risks. The paradigm allows a project to develop or extend a metrics program to address its specific risks. GQM has the following three Steps:

1. Generate a set of goals based upon the needs of the organization.
2. Derive a set of questions.
3. Develop a set of metrics, which provide the information needed to answer the questions.

The first step is to develop goals for the project and the management of risks. What is it that concerns the project? This is where the risks are initially identified.

The second step is to derive a set of questions that, if answered, would provide the status of the goals. The questions help quantify the goals and may give further details on the risks.

The third step is to develop a set of metrics that will answer the questions. Metrics can be used for both process and product related questions. Metrics will be discussed in the module on tracking risks.

GQM can be used as an iterative process. Initial goals tend to be rather generic. As risks are identified, goals/questions home in on specific areas of risk.

NASA Software Checklist

The NASA software checklist is organized by the project development phases, with emphasis on the software portion of the overall project lifecycle. NASA’s checklist can be used in the same manner as the TBQ to identify risks. It can be used in interviews, as a checklist, as a period review list, or in conjunction with the GQM paradigm. The following table is a list of possible examples of the generic risks that should be considered when any project contains software.

The table also contains practical questions that were gathered by experienced NASA engineers. The checklist is laid out with generic risks listed, followed by a column to indicate if it applies to the specific project. If it is partially a problem as stated, further clarification should be added. The last column is to indicate if the risk should be accepted or needs to be worked, i.e., the risk needs to be researched or mitigated.

Mil Std 338 Design Checklist - Partial

Mil Std 338 is used in a similar fashion as the taxonomy and the software checklist but contains questions on hardware design and reliability. This is only a partial list from Mil Std Handbook 338 identifying typical questions for a design review.

1. Is the design simple?
2. Minimum number of parts?
3. Are there adequate indicators to verify critical functions?
4. Are reliability requirements established for critical items?
5. Are standard high-reliability parts being used?
6. Have parts been selected to meet reliability requirements?
7. Are circuit safety margins ample?
8. Has provision been made for the use of electronic failure
9. Prediction techniques, including marginal testing?
10. Have normal modes of failure and magnitude of each mode for each item or critical part been identified?
11. Has redundancy been provided where needed to meet specified reliability?
12. Does the design account for early failure, useful life and wear out?

Table of Additional Methods and Tools

The Table Identify Methods and Tools summarize additional methods and tools for capturing statements of risk.

Table Identify Methods and Tools

Summary

There are many methods and approaches to identifying risk using the knowledge and uncertainty of the project personnel as input. Regardless of the approach, it is important to capture for each risk

• A risk statement in the condition-consequence format
• Context providing additional information on the risk

Looking at a list of all the risks provides a global picture of the risks on the project that may not be seen looking at each risk individually. (Note: This does not mean you necessarily have two different artifacts. They could be generated from a single set of data in a database.)

References

1. [Carr 93] Carr; Marvin, Konda; Suresh, Monarch; Ira; Ulrich; Carol, & Walker; Clay. Taxonomy-Based Risk Identification (CMU/SEI-93-TR-6, ADA266992). Pittsburgh, Pa.: Software Engineering Institute, Carnegie Mellon University, 1993. (Note: This reference is included in the guidebook reference list.)
2. [Gluch 94a]Gluch, David P.A. Construct for describing Software Development Risk (CMU/SEI-94-TR-14, ADA284922). Pittsburgh, PA.: Software Engineering Institute, Carnegie Mellon University, 1994.
3. Risk Management Plan Rational
4. Risk Management Database
   
5. The Software Engineering Information Repository (SEIR) is free, but you must be a registered member in order to access the information.