other versions
- wheezy 0.3-1
- wheezy-backports 0.12-1~bpo70+1
- jessie 0.12-1
- jessie-backports 0.27-1~bpo8+1
- testing 0.27-1
- unstable 0.27-1
CMDTEST(1) | General Commands Manual | CMDTEST(1) |
NAME¶
cmdtest - blackbox testing of Unix command line toolsSYNOPSIS¶
cmdtest [-c=COMMAND] [--command=COMMAND] [--config=FILE] [--dump-config] [--dump-memory-profile=METHOD] [--dump-setting-names] [--generate-manpage=TEMPLATE] [-h] [--help] [-k] [--keep] [--list-config-files] [--log=FILE] [--log-keep=N] [--log-level= LEVEL] [--log-max=SIZE] [--log-mode=MODE] [--no-default-configs] [--output=FILE] [-t=TEST] [--test=TEST] [--timings] [--version] [FILE]...DESCRIPTION¶
cmdtest black box tests Unix command line tools. Given some test scripts, their inputs, and expected outputs, it verifies that the command line produces the expected output. If not, it reports problems, and shows the differences. Each test case foo consists of the following files:- foo.script
- a script to run the test (this is required)
- foo.stdin
- the file fed to standard input
- foo.stdout
- the expected output to the standard output
- foo.stderr
- the expected output to the standard error
- foo.exit
- the expected exit code
- foo.setup
- a shell script to run before the test
- foo.teardown
- a shell script to run after test
- setup-once
- a shell script to run once, before any tests
- setup
- a shell script to run before each test
- teardown
- a shell script to run after each test
- teardown-once
- a shell script to run once, after all tests
—
execute setup
—
execute foo.setup
—
execute the command, by running foo.script, and redirecting standard
input to come from foo.stdin, and capturing standard output and error
and exit codes
—
execute foo.teardown
—
execute teardown
—
report result of test: does exit code match foo.exit, standard output
match foo.stdout, and standard error match foo.stderr?
• execute teardown-once Except for foo.script, all of these files are optional. If a setup or teardown script is missing, it is simply not executed. If one of the standard input, output, or error files is missing, it is treated as if it were empty. If the exit code file is missing, it is treated as if it specified an exit code of zero. The shell scripts may use the following environment variables:
- DATADIR
- a temporary directory where files may be created by the test
- TESTNAME
- name of the current test (will be empty for setup-once and teardown-once)
- SRCDIR
- directory from which cmdtest was launched
OPTIONS¶
- -c, --command=COMMAND
- ignored for backwards compatibility
- --config=FILE
- add FILE to config files
- --dump-config
- write out the entire current configuration
- --dump-memory-profile=METHOD
- make memory profiling dumps using METHOD, which is one of: none, simple, meliae, or heapy (default: simple)
- --dump-setting-names
- write out all names of settings and quit
- --generate-manpage=TEMPLATE
- fill in manual page TEMPLATE
- -h, --help
- show this help message and exit
- -k, --keep
- keep temporary data on failure
- --list-config-files
- list all possible config files
- --log=FILE
- write log entries to FILE (default is to not write log files at all); use "syslog" to log to system log, or "none" to disable logging
- --log-keep=N
- keep last N logs (10)
- --log-level=LEVEL
- log at LEVEL, one of debug, info, warning, error, critical, fatal (default: debug)
- --log-max=SIZE
- rotate logs larger than SIZE, zero for never (default: 0)
- --log-mode=MODE
- set permissions of new log files to MODE (octal; default 0600)
- --no-default-configs
- clear list of configuration files to read
- --output=FILE
- write output to FILE, instead of standard output
- -t, --test=TEST
- run only TEST (can be given many times)
- --timings
- report how long each test takes
- --version
- show program's version number and exit
EXAMPLE¶
To test that the echo(1) command outputs the expected string, create a file called echo-tests/hello.script containing the following content:-
#!/bin/sh echo hello, world
- hello, world
-
$ cmdtest echo-tests test 1/1 1/1 tests OK, 0 failures
-
$ cmdtest echo-tests FAIL: hello: stdout diff: --- echo-tests/hello.stdout 2011-09-11 19:14:47 +0100 +++ echo-tests/hello.stdout-actual 2011-09-11 19:14:49 +0100 @@ -1 +1 @@ -something else +hello, world test 1/1 0/1 tests OK, 1 failures