.TH adt\-run 1 2014 autopkgtest .SH NAME adt\-run \- test an installed binary package using the source package's tests .SH SYNOPSYS .B adt\-run .IR options ... .B \-\-\- .I virt\-server .RI [ virt\-server\-arg ...] .br .SH DESCRIPTION .B adt\-run is the program for invoking the autopkgtest package testing machinery. autopkgtest is a facility for testing binary Debian or Click packages, as installed on a system (such as a testbed system). The tests are those supplied in the source package. adt\-run runs each test supplied by a particular package and reports the results. It drives the specified virtualisation regime as appropriate, and parses the test description metadata, and arranges for data to be copied to and from the testbed as required. See /usr/share/doc/autopkgtest/README.running\-tests.rst.gz for an introduction about how to use adt\-run. .SH SPECIFYING TESTS Actions specify the source and binary packages to test, or change what happens with package arguments: .TP .BR --source " " \fIdsc\fR Run tests from Debian source package \fIdsc\fR. By default the package will also be built and the resulting binaries will be used to satisfy test dependencies; to disable that, specify the .BR -B / --no-built-binaries option before. The ordering is significant: each \fB--source\fR option should precede options whose dependencies are to be satisfied by the binaries it produces. .TP .BR --unbuilt-tree " " \fIdirectory\fR Specifies that tests from the unbuilt Debian source tree .IR directory should be run. This is very similar to specifying \fB\-\-source\fR except that a directory tree (which should be pristine) is supplied, instead of a source package. .TP .BR --built-tree " " \fIdirectory\fR Specifies that tests from the built Debian source tree .IR directory should be run. Note that all test dependencies are then satisfied by archive packages, unless you explicitly specify locally built .debs with .BR --binary . .TP .BR --apt-source " " \fIsrcpkgname\fR Downloads \fIsrcpkgname\fR with \fBapt\-get source\fR in the testbed and run its tests. This is similar to specifying .B \-\-source but avoids copying the source from the host to the testbed. Possibly built binaries will .B not be used to satisfy dependencies, as usually in this mode you want to test binaries from a real archive. .TP .BR --binary " " \fIdeb\fR Specifies that \fIdeb\fR should be used for tests of all following source packages. By default it will be used to satisfy dependencies, both during building and testing. The ordering is significant, as for \fB--source\fR. In particular, if a subsequent source package will build a binary of the same name, that will be used from then on, and \fIdeb\fR will be ignored. .TP .BR --changes " " \fIchanges\fR Specifies that the debs in the given .changes should be used for tests of the source package in that .changes. Acts as if you had specified the .debs and .dsc from a .changes file as explicit arguments. .TP .BR --click-source " " \fIclicksrc Path to click source tree for subsequent .B --click package. .TP .BR --click " " \fIclickpkg If .I clickpkg is a file (*.click), install given click package into testbed. If it is a click name (like "com.example.myapp"), assume it is already installed in the testbed and read the manifest from it. Run click package tests from the preceeding .BR --click-source . If a click source directory is not specified explicitly, it will be downloaded according to the manifest's .B x-source entry. Currently the only supported schema is .BR vcs-bzr . .TP .I filename Bare filename arguments are processed as if .BR --built-tree ", " --source ", " --unbuilt-tree ", " --apt-source ", " .BR --binary ", " --changes ", " --click-source ", or " --click was specified; the nature of the argument is guessed from the form of the filename. In the case of \fB--built-tree\fR, either the option must be specified, or the filename must end in a slash; two slashes at the end are taken to mean \fB--unbuilt-tree\fR. If a given directory has a "click" subdirectory, it is interpreted as .BR --click-source . .SH TEST OPTIONS Unless stated otherwise, these affect all subsequent test arguments. .TP .BR -B " | " --no-built-binaries All built binaries from subsequent .B --source or .B --unbuilt-tree tests will not be built or ignored, and dependencies are satisfied with packages from the archive. Note that packages still get built if a test requires \fBbuild-needed\fR. .TP .B --built-binaries Subsequent .B --source or .B --unbuilt-tree tests will get built and their dependencies be satisfied with the built binaries. This is the default behaviour, so you only need this to revert a previously specified .B --no-built-binaries option. .TP .BI --control-override= PATH Read the test metadata from .I PATH instead of .B debian/tests/control (for Debian sources) or the Click manifest for the following test. .SH LOGGING OPTIONS .TP .BI -o " dir" " | --output-dir=" dir Specifies that test artifacts (stderr and stdout from the tests, the log file, built binary packages etc.) should be placed in the given directory. \fIdir\fR will be created if necessary, and \fIemptied of all of its contents\fR before \fBadt-run\fR starts. .TP .BI -l " logfile" " | --log-file=" logfile Specifies that the trace log should be written to \fIlogfile\fR instead of to \fIoutput-dir\fR. .TP .BI --summary= summary Specifies that a summary of the outcome should be written to \fIsummary\fR. The events in the summary are written to the log in any case. .TP .BR -q " | " --quiet Do not send a copy of \fBadt-run\fR's trace logstream to stderr. This option does not affect the copy sent to \fIlogfile\fR or \fIoutput-dir\fR. Note that without the trace logstream it can be very hard to diagnose problems. .SH TEST BED SETUP OPTIONS .TP .BI \-\-setup\-commands= commands Run .I commands after opening the testbed. This can be used e. g. to enable additional apt sources, run .B apt-get update or similar. If .I commands is an existing file name, the commands are read from that; otherwise it is a string with the actual commands that gets run as-is. File names without directory will be searched in both the current directory and in .B /usr/share/autopkgtest/setup-commands/ so you do not need to give the full path for setup scripts shipped with autopkgtest. This option can be specified multiple times. If .B \-\-user is given or the test bed provides a .B suggested-normal-user capability, the .B $ADT_NORMAL_USER environment variable will be set to that user. If the setup commands affect anything in boot directories (like /boot or /lib/systemd/system) and the testbed supports rebooting, the testbed will be rebooted after the setup commands. This can be suppressed by creating a file .BR /run/autopkgtest_no_reboot.stamp . .TP .BR --apt-upgrade " | " -U Run .B apt\-get update and .B apt\-get dist-upgrade -y in the testbed before running the tests. .TP .BI \-\-apt\-pocket= pocket Add apt sources for \fIrelease\fR-\fIpocket\fR. This finds the first .B deb line in .B /etc/apt/sources.list which does not already specify a pocket and adds a deb and deb-src line with that pocket to .B /etc/apt/sources.list.d/\fIpocket\fB.list\fR. Note that this does not imply calling .B apt-get update\fR. .TP .BI \-\-copy= HOSTPATH:TESTBEDPATH Copy file or directory from host into testbed after opening. This happens before .B \-\-setup-commands thus you can use these files in the setup commands. .SH USER/PRIVILEGE HANDLING OPTIONS .TP .BI -u " user" " | --user=" user Run builds and tests as \fIuser\fR on the testbed. This needs root on the testbed; if root on the testbed is not available then builds and tests run as whatever user is provided. .TP .BI --gain-root= gain-root Prefixes .B debian/rules binary with .RB gain-root . The default is not to use anything, except that if \fB--user\fR is supplied or root on the testbed is not available the default is \fBfakeroot\fR. .SH DEBUGGING OPTIONS .TP .BR --debug | -d Include additional debugging information in the trace log. Each additional \fB-d\fR increases the debugging level; the current maximum is \fB-ddd\fR. If you like to see what's going on, \fR-d\fB or \fR-dd\fB is recommended. .TP .BR --shell-fail | -s Run an interactive shell in the testbed after any failed build or test. .TP .BR --shell Run an interactive shell in the testbed after every test. .SH TIMEOUT OPTIONS .TP .BR --timeout- \fIwhich\fR = \fIseconds\fR Use a different timeout for operations on or with the testbed. There are five timeouts affected by five values of \fIwhich\fR: .BR short : supposedly short operations like setting up the testbed's apt and checking the state (default: 100s); .BR install : installation of packages including dependencies (default: 3,000s); .BR test : test runs (default: 10,000s); .BR copy : copy files/directories between host and testbed (default: 300s); and .BR build : builds (default: 100,000s). The value must be specified as an integer number of seconds. .TP .BR --timeout-factor =\fIdouble\fR Multiply all of the default timeouts by the specified factor (see \fB--timeout-\fR\fIwhich\fR above). Only the defaults are affected; explicit timeout settings are used exactly as specified. .SH LOCALE OPTIONS .TP .BI --set-lang= langval When running commands on the testbed, sets the \fBLANG\fR environment variable to \fIlangval\fR. The default in \fBadt-run\fR is to set it to \fBC.UTF-8\fR. .TP .BI --leave-lang Suppresses the setting by \fBadt-run\fR of \fBLANG\fR on the testbed. This results in tests and builds using the testbed's own normal \fBLANG\fR value setting. .SH OTHER OPTIONS .TP .BI --gnupg-home= dir Uses \fIdir\fR as the gnupg key directory for local apt archive signing. The specified directory should not contain keyrings containing other unrelated keys, since \fBadt-run\fR does not specify to \fBgpg\fR which keys to use. The default is .BR $HOME/.cache/autopkgtest . .TP .BR \-h | \-\-help Show command line help and exit. .SH VIRTUALIZATION SERVER .TP \fB---\fR \fIvirt-server virt-server-arg\fR... Specifies the virtualisation regime server, as a command and arguments to invoke. .I virt-server must be an existing autopkgtest virtualization server such as .B adt-virt-schroot or .BR adt-virt-qemu . You can leave out the .B adt-virt- prefix and just specify the last part, e. g. .BR schroot . All the remaining arguments and options after .B --- are passed to the virtualisation server program. See the manpages of the individual servers for how to use them. .SH OUTPUT FORMAT During a normal test run, one line is printed for each test. This consists of a short string identifying the test, some horizontal whitespace, and either .B PASS or .BR FAIL " reason" or .BR SKIP " reason" where the pass/fail indication is separated by any reason by some horizontal whitespace. The string to identify the test consists of a short alphanumeric string invented by \fBadt-run\fR to distinguish different command-line arguments, the \fIargid\fR, followed by a hyphen and the test name. Sometimes a .B SKIP will be reported when the name of the test is not known or not applicable: for example, when there are no tests in the package, or a there is a test stanza which contains features not understood by this version of .BR adt-run . In this case .B * will appear where the name of the test should be. If \fBadt-run\fR detects that erroneous package(s) are involved, it will print the two lines .BR "blame: " \fIblamed-thing\fR ... and .BR "badpkg: " \fImessage\fR. Here each whitespace-separated \fIblamed-thing\fR is one of .BI arg: argument (representing a pathname found in a command line argument), .BI dsc: package (a source package name), .BI deb: package (a binary package name) or possibly other strings to be determined. This indicates which arguments and/or packages might have contributed to the problem; the ones which were processed most recently and which are therefore most likely to be the cause of a problem are listed last. .SH CONFIGURATION FILES If you use lots of options or nontrivial virt server arguments, you can put any part of the command line into a text file, with one line per option. E. g. you can create a file .I sid.cfg with contents like .RS .EX -s --output-dir=/tmp/testout --apt-upgrade --- schroot sid .EE .RE and then run .RS .EX adt-run foo_1_amd64.changes @sid.cfg .EE .RE The contents of the configuration file will be expanded in-place as if you would have given its contents on the command line. Please ensure that you .B don't place spaces between short options and their values, they would become a part of the argument value. .SH EXIT STATUS 0 all tests passed .br 2 at least one test skipped .br 4 at least one test failed .br 6 at least one test failed and at least one test skipped .br 8 no tests in this package .br 12 erroneous package .br 16 testbed failure .br 20 other unexpected failures including bad usage .SH SEE ALSO \fB/usr/share/doc/autopkgtest/README.running-tests.rst.gz\fR .br \fB/usr/share/doc/autopkgtest/README.package-tests.rst.gz\fR .SH AUTHORS AND COPYRIGHT This manpage is part of autopkgtest, a tool for testing Debian binary packages. autopkgtest is Copyright (C) 2006-2014 Canonical Ltd. See \fB/usr/share/doc/autopkgtest/CREDITS\fR for the list of contributors and full copying conditions.