NAME¶
Covered - Verilog Code Coverage Analyzer
SYNTAX¶
covered [
global_options]
score [
options]
covered [
global_options]
merge [
options]
existing_database database_to_merge+
covered [
global_options]
report [
options]
database_file
covered [
global_options]
rank [
options]
database_to_rank database_to_rank+
covered [
global_options]
exclude [
options]
exclusion_id+
database_file
DESCRIPTION¶
Covered is a Verilog code coverage analysis tool that can be useful for
determining how well a diagnostic test suite is covering the design under
test. Covered reads in the Verilog design files and a VCD, LXT2 or FST
formatted dumpfile from a diagnostic run and generates a database file called
a Coverage Description Database (CDD) file, using the score command. Covered's
score command can alternatively be used to generate a CDD file and a Verilog
module for using Covered as a VPI module in a testbench which can obtain
coverage information in parallel with simulation (see
USING COVERED AS A
VPI MODULE). The resulting CDD file can be merged with other CDD files
from the same design to create accummulated coverage, using the merge command.
Once a CDD file is created, the user can use Covered to generate various
human-readable coverage reports in an ASCII format or use Covered's GUI to
interactively look at coverage results, using the report command. If uncovered
coverage points are found that the user wants to exclude from coverage, this
can be handled with either the command-line exclude command or within the GUI.
When multiple CDD files are created from the same design, the user may obtain
a coverage ranking of those CDD files to determine an ideal order for
regression testing as well as understand which CDD files can be excluded from
regressions due to their inability to hit new coverage points. Additionally,
as part of Covered's score command, race condition possibilities are found in
the design files and can be either ignored, flagged as warnings or flagged as
errors. By specifying race conditions as errors, Covered can also be used as a
race condition checker.
GLOBAL OPTIONS¶
These options are placed immediately after the keyword
covered in the
command-line. They can be used for any command (with the exception of
-v and
-h) and have the same effect in each case.
- -B
- Obfuscate. Obfuscates all design-sensitive names before
outputting in user-readable format. This option is useful when sharing
output with the developers of Covered for debugging purposes.
- -D
- Debug. Display information helpful for debugging tool
problems. Note: This option is now only available when covered is built
with the --enable-debug configuration option.
- -h
- Help. Display this usage information.
- -P [filename]
- Profiling mode. Turns on internal source code profiler that
will produce a profiling report of the run command to either the specified
filename or, if no filename is present, to a file called
covered.prof. This option is only available if the
--enable-profiling configuration option was specified when Covered
was built.
- -Q
- Quiet mode. Causes all output to be suppressed.
- -T
- Terse mode. Causes all output to be suppressed with the
exception of warning messages and the Covered header information.
- -v
- Version. Display current Covered version.
COMMANDS¶
- score
- Parses Verilog files and VCD/LXT2/FST dumpfiles to create
database file used for merging and reporting.
- merge
- Merges two or more database files into one.
- report
- Generates human-readable coverage reports from database
file or starts the coverage report GUI.
- rank
- Generates a report that specifies an ideal order to run
regressions and specifies CDD files that do not add new coverage
information (and can, therefore, be excluded from regressions, if
desired).
- exclude
- Allows one or more coverage points (identified with
exclusion IDs in a report generated with the -x option) to have their
exclusion property toggled (include to exclude or exclude to include) and,
if the exclusion property is set to exclude, optionally allow a reason for
the exclusion to be associated with it and stored in the specified CDD
file.
SCORE COMMAND¶
The following options are valid for the score command:
- -A ovl
- Causes OVL assertions to be used for assertion coverage.
This flag must be given to the score command if assertion coverage metrics
are needed in the report command.
- -cdd database
- Same as the -o option. Useful when CDD file being scored is
an input to the score command.
- -cli [filename]
- Causes the command-line debugger to be used during
VCD/LXT2/FST dumpfile scoring. If filename is specified, this file
contains information saved in a previous call to savehist on the CLI and
causes the history contained in this file to be replayed prior to the CLI
command prompt. If filename is not specified, the CLI prompt will
be immediately available at the start of simulation. This option is only
available when Covered is configured with the --enable-debug
option.
- -conservative
- If this option is specified, any logic blocks that contain
code that could cause coverage discrepancies leading to potentially
inaccurate coverage results are removed from coverage consideration. See
User's Guide for more information on what type of code can lead to
coverage inaccuracies.
- -dumpvars [filename]
- If this option is specified without the -vcd or -lxt
options, the design is parsed, a CDD file is created and a top-level
Verilog module file named filename (if this value is specified) or
"covered_dump.v" (if filename is not specified) is
created. This file is used in the compilation of the simulator to create a
dumpfile that is optimized for obtaining coverage for the specified CDD
file. If either the -vcd or -lxt options are specified, this option has no
effect. See the User's Guide for more information on how to use this
option.
- -D define_name
- Defines the specified name to 1.
- -D define_name=value
- Defines the specified name to the specified value.
- -e block_name
- Name of module, task, function or named begin/end block to
not score. Causes all subblocks in the Verilog tree under this block to
also not be scored.
- -ea
- Excludes all always blocks from being considered for
coverage.
- -ec
- Excludes all continuous assignments from being considered
for coverage.
- -ei
- Excludes all initial blocks from being considered for
coverage.
- -ef
- Excludes all final blocks from being considered for
coverage.
- -ep [name]
- Excludes all code found between '// coverage off' and '//
coverage on' pragmas embedded within the design. If name is
specified, the keyword in the pragma will be changed to that name instead
of the default "coverage", allowing the user to change the look
of the pragma if it conflicts with any other tools.
- -F module_name=[in_expr,]out_expr
- Indicates to the parser where to find the FSM located in
module module_name which has an input state expression called
in_expr and output state expression called out_expr. If
in_expr is not specified, out_expr is used as both the input
and output state expression.
- -f filename
- Name of file containing additional arguments to parse.
- -fst filename
- Name of FST dumpfile to score design with. If -vcd, -lxt or
this option is not used, Covered will only create an initial CDD file from
the design and will not attempt to score the design.
- -g [module_name=](1|2|3)
- Allows the user to limit the parser to a specific
generation of the Verilog standard for a specific module or the entire
design, where 1=Verilog-1995, 2=Verilog-2001, 3=SystemVerilog. If
module_name= is specified, the parser uses the specified Verilog
standard for that module only, allowing the rest of the design to use the
global standard. By default, the global standard is set to the most recent
version (3). This option can be specified more than once for a given call
to the score command; however, if the -g option specifies more than global
value (i.e., without the module= prefix), only the last option
value will be used.
- -h
- Displays this help information.
- -I directory
- Directory to find included Verilog files.
- -i instance_name
- Verilog hierarchical reference to the module that is at the
top of the tree to be scored. This option is necessary if module to verify
coverage is not the top-level module in the design. If not specified,
-t value is used.
- -lxt filename
- Name of LXT2 dumpfile to score design with. If -vcd, -fst
or this option is not used, Covered will only create an initial CDD file
from the design and will not attempt to score the design.
- -m message
- Allows the user to specify information about this CDD file.
This information can be anything (messages with whitespace should be
surrounded by double-quotation marks), but may include something about the
simulation arguments to more easily link the CDD file to its simulation
for purposes of recreating the CDD file.
- -o database
- Name of database to write coverage information to. If not
specified, the output database filename will be "cov.cdd".
- -p filename
- Overrides default filename used to store intermediate
preprocessor output.
- -P parameter_scope=value
- Performs a defparam on the specified parameter with
value.
- -rS
- When race condition checks are violated, the offending
logic blocks are removed from coverage consideration and all output is
suppressed regarding the race condition violation. See user documentation
for more information about race condition checking usage.
- -rW
- When race condition checks are violated, the offending
logic blocks are removed from coverage consideration and the race
condition violation is output. This is the default behavior for race
condition handling. See user documentation for more information about race
condition checking usage.
- -rE
- When race condition checks are violated, the reason is
output and scoring ends immediately. See user documentation for more
information about race condition checking usage.
- -rI[=module name]
- If module name is not specified, race condition
checking is skipped altogether for the entire design. If module
name is specified, race condition checking is skipped for the
specified module. See user documentation for more information about race
condition checking usage.
- -rP[=name])
- Uses embedded pragmas for ignoring certain code from race
condition checking consideration (if name is specified it is used
as the pragma keyword). See user documentation for more information about
race condition checking usage.
- -S
- Outputs simulation statistics after simulation has
completed. This information is currently only useful for the developers of
Covered.
- -t top-level module
- Specifies the module name of the top-most module that will
be measured. Note that this module does not need to be the top-most module
in the simulator. This field is required for all calls to the score
command.
- -top_ts timescale
- This option is only valid when the -vpi or -dumpvars
options have been specified. This option allows the user to specify a
timescale for the generated Verilog module created with the -vpi/-dumpvars
option. If this option is not specified, no timescale will be created for
the generated module. The value of timescale is specified as
follows:
- (1|10|100)(s|ms|us|ns|ps|fs)/(1|10|100)(s|ms|us|ns|ps|fs)
- If whitespace is needed between the various values, place
the entire contents of timescale in double quotes.
- -ts number
- When scoring occurs, this option allows the user to see how
far the simulator has progressed by outputting the current timestep to
standard output. The value of number specifies how many timesteps
are allowed to be simulated before outputting the current timestep
(results in less calls to output stream).
- -T (min|typ|max)
- Specifies which value to use when encountering a delay
expression in the form: min:typ:max. If this option is not specified,
'typ' select is used by default.
- -v filename
- Name of specific Verilog file to score.
- -vcd filename
- Name of VCD dumpfile to score design with. If -lxt, -fst or
this option is not used, Covered will only create an initial CDD file from
the design and will not attempt to score the design.
- -vpi [filename]
- If this option is specified without the -vcd, -lxt or -fst
options, the design is parsed, a CDD file is created and a top-level
Verilog module file named filename (if this value is specified) or
"covered_vpi.v" (if filename is not specified) is created
along with a PLI table file called filename.ta b or
"covered_vpi.v.ta b". Both of these files are used in the
compilation of the simulator to use Covered as a VPI module. If either the
-vcd, -lxt or -fst options are specified, this option has no effect.
- -Wignore
- Suppress the output of warnings during code parsing and
simulation.
- -y directory
- Directory to find unspecified Verilog files.
- +libext+.extension[+.extension]*+
- Extensions of Verilog files to allow in scoring.
MERGE COMMAND¶
The following options are valid for the merge command:
- -d filename
- Directory to search for CDD files to include. This option
is used in conjunction with the -ext option which specifies the
file extension to use for determining which files in the directory are CDD
files.
- -er
(first|last|all|new|old)
- Specifies how to handle exclusion reason resolution. If two
or more CDD files being merged have exclusion reasons specified for the
same coverage point, the exclusion reason needs to be resolved (unless it
is the same string value). If this option is not specified and a conflict
is found, Covered will interactively request input for each exclusion as
to how to handle it. If this option is specified, it tells Covered how to
handle all exclusion reason conflicts. The values are as follows:.br
- first - CDD file that contained the first exclusion
reason is used.
- last - CDD file that contained the last exclusion
reason is used.
- all - All exclusion reasons are used
(concatenated).
- new - Use the newest exclusion reason
specified.
- old - Use the oldest exclusion reason
specified.
- -ext extension
- Used in conjunction with the -d option. If no
-ext options are specified on the command-line, the default value
of '.cdd' is used. Note that a period (.) should be specified.
- -f filename
- Name of file containing additional arguments to parse.
- -h
- Displays this help information.
- -m message
- Allows the user to specify information about this CDD file.
This information can be anything (messages with whitespace should be
surrounded by double-quotation marks).
- -o filename
- File to output new database to. If this argument is not
specified, the existing_database is used as the output database
name.
REPORT COMMAND¶
The following options are valid with the report command:
- -b
- If combinational logic verbose output is reported and the
expression is a vector operation, this option outputs the coverage
information on a bitwise basis.
- -c
- If -v is specified, displays covered metrics only.
Default is to display uncovered information only.
- -d (s|d|v)
- Level of detail to provide in coverage report information
(s = summary, d = detailed, v = verbose). Default is summary.
- -e
- Outputs all excluded coverage points to the report file
along with any specified exclusion reasons if the -d d or -d
v options are specified.
- -f filename
- Name of file containing additional arguments to parse.
- -h
- Displays this help information.
- -i
- Provides coverage information for instances instead of
module.
- -m [l][t][c][f][r][a][m]
- Type(s) of metrics to report. l=line, t=toggle,
c=combinational logic, f=FSM state and state transition, r=race
conditions, a=assertion, m=memory. Default is ltcf.
- -o filename
- File to output report information to. Default is standard
output.
- -s
- Suppresses modules/instances that contain no coverage
information from being output to the report. Used to help eliminate
potentially meaningless information from the report.
- -v
- Deprecated. Replaced by '-d d' or '-d v'.
- -view
- Starts the GUI interface for interactive coverage
reporting.
- -w (number)
- Specifies the maximum line width (in characters) that can
be used to output Verilog information. If this option is not specified,
all Verilog code in the report will retain the same formatting as was
specified in the original Verilog code. If this option is specified,
Verilog code will be formatted to use as much of the current line as
possible, wrapping text when the line reaches the maximum line width. The
default maximum line width is 115 characters (this value is used if no
number is specified with the -w option). If a number is specified with the
-w option, this value is used for the maximum line width.
- -x
- Outputs the exclusion IDs of all uncovered and excluded
coverage points within parenthesis before the associated verbose output of
the coverage point. The exclusion IDs can be used to exclude/include
coverage points via the exclude command.
RANK COMMAND¶
The following options are valid with the rank command:
- -d filename
- Directory to search for CDD files to include. This option
is used in conjunction with the -ext option which specifies the
file extension to use for determining which files in the directory are CDD
files.
- -depth number
- Specifies the minimum number of needed CDD files to hit
each coverage point. The value of number should be greater than
zero. Default is 1.
- -ext extension
- Used in conjunction with the -d option. If no
-ext options are specified on the command-line, the default value
of '.cdd' is used. Note that a period (.) should be specified.
- -f filename
- Name of file containing additional arguments to parse.
- -h
- Displays help information for the rank command.
- -names-only
- If specified, outputs only the needed CDD filenames that
need to be run in the order they need to be run. If this option is not
set, a report-style output is provided with additional information. This
option is meant to be useful in scripts that only want CDD filenames to
run as output.
- -o filename
- Name of file to output ranking information to. Default is
standard output.
- -required-cdd filename
- Name of CDD that should be considered a required CDD to
rank (i.e., it cannot be excluded for any reason).
- -required-list filename
- Name of a file that contains a list of CDDs that should be
considered required CDDs to rank. The filenames should be separated by
whitespace or newline characters within the file.
- -v
- Causes verbose output to be displayed when the rank command
is run. It outputs diagnostic information about each of the different
phases of the ranking algorithm including run-time, number of CDD files
included/excluded and number of coverage points hit by ranked CDDs during
each phase. This information is meant to be useful for those interested in
the ranking algorithm and its performance.
- -weight-assert number
- Specifies a relative weighting for assertion coverage used
to rank non-unique coverage points. The value of number is relative
to the values used in the -weight-toggle, -weight-memory,
-weight-comb, -weight-fsm and -weight-line rank
command options.
- -weight-comb number
- Specifies a relative weighting for combinational logic
coverage used to rank non-unique coverage points. The value of
number is relative to the values used in the -weight-toggle,
-weight-memory, -weight-assert, -weight-fsm and
-weight-line rank command options.
- -weight-fsm number
- Specifies a relative weighting for FSM state and state
transition coverage used to rank non-unique coverage points. The value of
number is relative to the values used in the -weight-toggle,
-weight-memory, -weight-comb, -weight-assert and
-weight-line rank command options.
- -weight-line number
- Specifies a relative weighting for line coverage used to
rank non-unique coverage points. The value of number is relative to
the values used in the -weight-toggle, -weight-memory,
-weight-comb, -weight-fsm and -weight-assert rank
command options.
- -weight-memory number
- Specifies a relative weighting for memory coverage used to
rank non-unique coverage points. The value of number is relative to
the values used in the -weight-toggle, -weight-line,
-weight-comb, -weight-fsm and -weight-assert rank
command options.
- -weight-toggle number
- Specifies a relative weighting for toggle coverage used to
rank non-unique coverage points. The value of number is relative to
the values used in the -weight-memory, -weight-line,
-weight-comb, -weight-fsm and -weight-assert rank
command options.
EXCLUDE COMMAND¶
The following options are valid with the exclude command:
- -f filename
- Specifies the name of a file that contains more options to
the exclude command. This option may be specified as many times as
necessary for a single call to the exclude command.
- -h
- Generates usage information for the exclude command.
- -m
- Allows for an exclusion message to be associated with any
coverage points going from the included state to the excluded state. For
each coverage point that meets this requirement, the user will be prompted
to input a reason. The reason may be any length and any number of lines;
however, all formatting characters (i.e., newlines, tabs, extra spaces,
etc.) will be removed and replaced with a single space when it is later
displayed. To end the input of a message, hit a return, enter a single
period (.) character and hit return again. The final period character will
not be part of the exclusion message.
- -p
- Causes all specified coverage points to print their current
exclusion status and exclusion reason (if one exists for the excluded
coverage point) to standard output. If this option is specified, the
-m option will be ignored.
USING COVERED AS A VPI MODULE¶
- In addition to using Covered's score command to parse a
VCD, LXT2 or FST file to abstract coverage information, Covered may also be
used as a VPI module within a simulator to extract this information. The
advantages to using Covered as a VPI over a dumpfile reader include the
following. First, VCD files can be extrememly large, especially for long
simulations, using up valuable disk space. Second, if you are using a
simulator that dumps files in a different format than VCD, LXT2 or FST and
you want to convert these dump file types to one of these versions, the cost
of disk space and time can make creating dumpfiles that Covered requires
undesirable. Additionally, though using Covered as a VPI module will slow
down your simulation speed, it is most likely that the total time spent
simulating your design and scoring the design in one step will be shorter
than doing so in two steps. As a result, Covered's configure utility can
generate VPI-ready libraries for the following free and commercial
simulators (Icarus Verilog, CVER and VCS).
- To automatically build the VPI-ready library files when
generating Covered from source, simply specify one or more of the following
when running the "configure" utility in the base Covered
directory: --with-iv=<Icarus Verilog install path>, --with-vcs=<VCS
include path>, --with-cver=<CVER include path>. After Covered has
been configured, simply type 'make' and 'make install'. This will install
the VPI-ready library files in the installation libexec directory (by
default this path will be /usr/local/libexec).
- Before you are ready to compile the design, you must first
create a CDD file, a top-level Verilog file, and a PLI table file (the last
file is only needed for the VCS compiler). This is done by specifying the
-vpi (filename) option to Covered's score command. If no
filename is specified after -vpi, the files covered_vpi.v and covered_vpi.ta
b will be created along with the generated coverage file. Note that this
step only needs to be performed once unless the design files change. You are
now ready to compile the simulator.
- If you are compiling an Icarus Verilog simulation, simply
add '-m /usr/local/libexec/covered.vpi covered_vpi.v' to the 'iverilog'
command-line. Once compilation is complete, run the generated executable
file as you normally would.
- If you are compiling a CVER simulation, simply add
'+loadvpi=/usr/local/libexec/covered.cver.so:vpi_compat_bootstrap
covered_vpi.v' to the 'cver' command-line.
- If you are compiling a VCS simulation, simply add '+vpi
-load /usr/local/libexec/covered.vcs.so:covered_register covered_vpi.v' to
the 'vcs' command-line. Once compilation is complete, run the generated
executable file as you normally would.
- If you are compiling a NC-Verilog simulation, switch to
NC-Verilog's irun command to load the covered shared object: '-loadvpi
/usr/local/libexec/covered.ncv.so:covered_register' and enable all access
with '-access +rwc'. You can hardcode the $covered_sim call into your RTL or
you can run it dynamically using the CLI, by adding the -input input.tcl
switch to irun. Where the input.tcl file looks like the following and tb.dut
is the coverage instance:
-
call -systf {$covered_sim} {"scored.cdd"} tb.dut
run
- There are two plusargs that can be passed to the generated
executable when it is run that Covered will parse. The
'+covered_cdd=<filename>' option will cause Covered to output the
scored design contents to the CDD file specified by <filename>. This
allows multiple runs of the simulator to generate several different CDD
files without needed a recompile to occur. The '+covered_debug' option will
cause Covered to dump a lot of excessive output about its internal run-time
state during simulation. This output will only be generated if Covered was
configured with the --enable-debug option. This plusarg option should not be
used by regular users as it is primarily intended to aid the developers of
Covered in debugging.
AUTHORS¶
Trevor Williams <phase1geo@gmail.com>
SEE ALSO¶
For more information on how to use the Covered code coverage tool, please
consult the User's Guide included with this release at
/usr/local/share/covered/doc/html/index.html.