Tool Mentor:  Evaluating Code Coverage Using Rational PureCoverage and Rational PurifyPlus (UNIX)

Purpose

This tool mentor provides an overview of how to use Rational PureCoverage® to quickly and easily identify the gaps in your testing.

PurifyPlus is a Rational product that includes PureCoverage functionality. 

To learn more about PureCoverage and for a tutorial-style introduction, read the Installing and Getting Started manual for the PurifyPlus product family (UNIX version).

For step-by-step instructions on using PureCoverage and for reference information, see the PureCoverage online Help. 

Related Rational Unified Process information:

• Activity: Perform Unit Tests, refer to the section titled "Execute Unit Test".
• Activity: Analyze Execution Failures, refer to the section titled "Evaluate Execution of Test"
• Activity: Determine Test Results, refer to the section titled "Evaluate Code-based Test Coverage"

Overview

PureCoverage monitors your program as it runs, and reports what functions and lines have and have not been exercised. You can use PureCoverage on any scale: you can collect data for an informal test of a single program functionality, or combine data from all the runs controlled by your test suite. Used with Rational Purify, PureCoverage can tell you what parts of your program have not been checked for memory errors and leaks, and so helps you improve significantly the quality of the code you ship to customers. 

Tool Steps

To assess and improve the effectiveness of your test coverage:

1. Run a program under PureCoverage to collect coverage data
2. Use PureCoverage to analyze the coverage data
3. Use your analysis to retest your program more thoroughly

1. Run a program under PureCoverage to collect coverage data 

The first step in improving coverage is to collect the code coverage data.

Just add the word purecov to the beginning of your compile/link line. PureCoverage collects coverage data at the function level and, if you compile with the -g debugging option, at the line level. For example:

% purecov cc -g hello_world.c

PureCoverage makes a copy of your object code and libraries, and instruments them using Object Code Insertion (OCI) technology to insert instructions to detect which functions and lines are executed when you run the program. When instrumentation is complete, run the instrumented program. For example:

% a.out

As you exercise your code, PureCoverage records data about code coverage.

You can call PureCoverage API functions from your program to take a snapshot of the current data, and then examine code coverage in stages.

When you exit the program, you can display the coverage data in the PureCoverage Viewer by typing:

% purecov -view a.out.pcv

The PureCoverage Viewer summarizes coverage information for all files and functions in your program, as well as detailed information for individual files, functions, and lines, arranged hierarchically so you can easily focus on the critical parts of your code.

Tip: In addition to using PureCoverage interactively, you can also incorporate it into your test scripts for automated testing. 

For more information, look up the following topics in the PureCoverage online Help index:

• basic steps
• API functions
• report scripts

2. Use PureCoverage to analyze the coverage data 

The second step in improving coverage is to analyze the code coverage data that PureCoverage collected.

The PureCoverage Viewer provides information such as the number of functions and the number of lines hit and missed by your tests. In addition, you can see the percentage of functions and lines that were hit and the number of times each function was called. You can also use the Annotated Source window to further examine the program's code coverage, and to examine a specific function's coverage line-by-line.

To reduce the volume of coverage data you collect and display, and to improve runtime performance, you can exclude files, or entire directories, when you instrument your program.

Actual coverage data is the basis for PureCoverage's statistics and annotated source display. However, there are occasions when it is more useful to adjust the data to exclude specific lines, such as code that is logically unavailable or is extremely difficult to reach. Using PureCoverage you can mark code as dead code, tested code, and inspected code. This keeps the code from being counted as unused code.

By default, PureCoverage merges the results of multiple runs of a program so you can easily analyze the total coverage of the program from a series of tests. You can also merge specific runs yourself, or keep the coverage data from multiple runs in separate files to analyze changes in coverage.

For more information, look up the following topics in the PureCoverage online Help index:

• PureCoverage viewer
• annotated source window
• excluding files form coverage
• adjusting coverage

3. Use your analysis to retest your program more thoroughly 

The third and final step in improving coverage is to use your coverage analysis to modify your tests and exercise previously untested lines and functions, and then to use PureCoverage to verify the improvements.

Using the modified tests, rerun your program under PureCoverage. Use the pc_diff report script to compare the new results to the baseline dataset you collected for your initial analysis.

PureCoverage saves coverage data as a data file (.pcv) that you can use for further analysis or share with other PureCoverage users. You can save data to an ASCII text file (.txt) to use outside of PureCoverage or you can format the data on your own by using the provided report scripts.

To track your code coverage from release to release, archive your saved .pcv files with your product releases.

Tip: If you have incorporated PureCoverage into your test scripts, you can automatically save coverage data to a text file (.txt).  

For more information, look up the following topics in the PureCoverage online Help index:

• pc_diff
• saving data

Copyright  © 1987 - 2001 Rational Software Corporation

Rational Unified Process