.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.40)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is >0, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{\
.    if \nF \{\
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{\
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\"
.\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2).
.\" Fear.  Run.  Save yourself.  No user-serviceable parts.
.    \" fudge factors for nroff and troff
.if n \{\
.    ds #H 0
.    ds #V .8m
.    ds #F .3m
.    ds #[ \f1
.    ds #] \fP
.\}
.if t \{\
.    ds #H ((1u-(\\\\n(.fu%2u))*.13m)
.    ds #V .6m
.    ds #F 0
.    ds #[ \&
.    ds #] \&
.\}
.    \" simple accents for nroff and troff
.if n \{\
.    ds ' \&
.    ds ` \&
.    ds ^ \&
.    ds , \&
.    ds ~ ~
.    ds /
.\}
.if t \{\
.    ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u"
.    ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'
.    ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'
.    ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'
.    ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'
.    ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'
.\}
.    \" troff and (daisy-wheel) nroff accents
.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'
.ds 8 \h'\*(#H'\(*b\h'-\*(#H'
.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#]
.ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'
.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'
.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#]
.ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#]
.ds ae a\h'-(\w'a'u*4/10)'e
.ds Ae A\h'-(\w'A'u*4/10)'E
.    \" corrections for vroff
.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'
.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'
.    \" for low resolution devices (crt and lpr)
.if \n(.H>23 .if \n(.V>19 \
\{\
.    ds : e
.    ds 8 ss
.    ds o a
.    ds d- d\h'-1'\(ga
.    ds D- D\h'-1'\(hy
.    ds th \o'bp'
.    ds Th \o'LP'
.    ds ae ae
.    ds Ae AE
.\}
.rm #[ #] #H #V #F C
.\" ========================================================================
.\"
.IX Title "Lintian::Tutorial::TestSuite 3"
.TH Lintian::Tutorial::TestSuite 3 "2022-07-02" "Lintian v2.115.1~bpo11+1" "Debian Package Checker"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
Lintian::Tutorial::TestSuite \-\- Quick intro to running the Lintian testsuite
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
Warning: This document may be out of date.
.PP
This guide will quickly introduce you to running the Lintian test
suite and some tricks.  The Lintian test suite is fairly large and
accordingly it can take a substantial amount of time to run.  To speed
up development, there are various options to limit the tests to run.
.PP
If you are looking for a guide on how to write tests, please consult
Lintian::Tutorial::WritingTests.
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
The Lintian test suite is an extensive collection of various test
cases.  The test suite is divided into 4 \*(L"sub-suites\*(R".  The majority
of tests are currently located in the \*(L"tests\*(R" sub-suite.
.PP
To run the full suite:
.PP
.Vb 1
\& $ rm \-rf debian/test\-out; private/build\-test\-packages; private/runtests
.Ve
.PP
While writing a new tag (or check) you probably only want to run a
particular (subset of the) test(s).  See \*(L"Running a subset of the
tests\*(R" for the available options.
.SS "Running a subset of the tests"
.IX Subsection "Running a subset of the tests"
First, you have to build the test packages with:
.PP
.Vb 1
\& $ rm \-rf debian/test\-out; private/build\-test\-packages;
.Ve
.PP
Then, the following options are available:
.IP "Running a single test" 4
.IX Item "Running a single test"
To run a single test by its name, use:
.Sp
.Vb 1
\& $ private/runtests \-\-onlyrun=test:$name
.Ve
.IP "Running all tests for a check" 4
.IX Item "Running all tests for a check"
To run all tests for a given check, use:
.Sp
.Vb 1
\& $ private/runtests \-\-onlyrun=check:$name
.Ve
.Sp
\&\f(CW$check\fR must be the name of a check (it will test for
checks/$check.desc) or \*(L"legacy\*(R".  This will run all tests that start
with \*(L"$check\-\*(R".
.IP "Running all tests designed for a specific tag" 4
.IX Item "Running all tests designed for a specific tag"
To run all tests that have a \*(L"Test-For\*(R" or a \*(L"Test-Against\*(R" for a given
tag, use:
.Sp
.Vb 1
\& $ private/runtests \-\-onlyrun=tag:$name
.Ve
.SS "Running tests under coverage"
.IX Subsection "Running tests under coverage"
This feature is currently untested.
.PP
It is possible to run most of the tests under Devel::Cover.  This is
done by passing \fI\-\-coverage\fR to \fIprivate/runtests\fR.  Example:
.PP
.Vb 1
\&  $ private/runtests \-\-coverage \-\-dump\-logs \-j1 \-k t debian/test\-out
.Ve
.PP
Please note that Devel::Cover does not seem to handle multiple
threads too well.  You may see spurious warnings/errors if you run the
tests with 2 or more active worker threads.
.PP
\&\fBCaveat\fR 1: Coverage for collections (i.e. programs in \fIcollection/\fR)
does not seem to work at the moment. Therefore, they often end up with
(next to) zero coverage in the generated reports.
.PP
\&\fBCaveat\fR 2: Devel::Cover sometimes changes the output of Lintian
or tools called by Lintian. Obviously, this leads to test
failures. Therefore, you may see weird test failures (or warnings)
when running under coverage.
.PP
\fICollecting the coverage you want in a reasonable time\fR
.IX Subsection "Collecting the coverage you want in a reasonable time"
.PP
Collecting coverage is excruciatingly slow.  This is not helped by the
fact that it becomes unreliable when run under 2 or more threads.
.PP
Fortunately, Devel::Cover \*(L"appends\*(R" to its cover database.  This
allows you to \*(L"slowly\*(R" build up the coverage database over multiple
runs. Example:
.PP
.Vb 4
\&  $ private/runtests \-\-coverage \-\-dump\-logs \-j1 \-k t debian/test\-out suite:scripts
\&  $ private/runtests \-\-coverage \-\-dump\-logs \-j1 \-k t debian/test\-out suite:debs
\&  $ private/runtests \-\-coverage \-\-dump\-logs \-j1 \-k t debian/test\-out suite:source
\&  ...
.Ve
.PP
Or:
.PP
.Vb 2
\&  $ private/runtests \-\-coverage \-\-dump\-logs \-j1 \-k t debian/test\-out $check
\&  $ private/runtests \-\-coverage \-\-dump\-logs \-j1 \-k t debian/test\-out legacy
.Ve
.SH "SEE ALSO"
.IX Header "SEE ALSO"
Lintian::Tutorial::WritingTests