.TH common_test 3erl "common_test 1.8.2" "Ericsson AB" "Erlang Module Definition"
.SH NAME
common_test \- A framework for automated testing of arbitrary target nodes
.SH DESCRIPTION
.LP
The \fICommon Test\fR\& framework is an environment for implementing and performing automatic and semi-automatic execution of test cases\&. Common Test uses the OTP Test Server as engine for test case execution and logging\&.
.LP
In brief, Common Test supports:
.RS 2
.TP 2
*
Automated execution of test suites (sets of test cases)\&.
.LP
.TP 2
*
Logging of the events during execution\&.
.LP
.TP 2
*
HTML presentation of test suite results\&.
.LP
.TP 2
*
HTML presentation of test suite code\&.
.LP
.TP 2
*
Support functions for test suite authors\&.
.LP
.TP 2
*
Step by step execution of test cases\&.
.LP
.RE

.LP
The following sections describe the mandatory and optional test suite functions Common Test will call during test execution\&. For more details see \fBCommon Test User\&'s Guide\&.\fR\& 
.SH "TEST CASE CALLBACK FUNCTIONS"

.LP
The following functions define the callback interface for a test suite\&.
.SH EXPORTS
.LP
.B
Module:all() -> Tests | {skip,Reason} 
.br
.RS
.LP
Types:

.RS 3
Tests = [TestCase | {group,GroupName} | {group,GroupName,Properties} | {group,GroupName,Properties,SubGroups}]
.br
TestCase = atom()
.br
GroupName = atom()
.br
Properties = [parallel | sequence | Shuffle | {RepeatType,N}] | default
.br
SubGroups = [{GroupName,Properties} | {GroupName,Properties,SubGroups}]
.br
Shuffle = shuffle | {shuffle,Seed}
.br
Seed = {integer(),integer(),integer()}
.br
RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail
.br
N = integer() | forever
.br
Reason = term()
.br
.RE
.RE
.RS
.LP
MANDATORY
.LP
This function must return the list of all test cases and test case groups in the test suite module that are to be executed\&. This list also specifies the order the cases and groups will be executed by Common Test\&. A test case is represented by an atom, the name of the test case function\&. A test case group is represented by a \fIgroup\fR\& tuple, where \fIGroupName\fR\&, an atom, is the name of the group (defined in \fB\fIgroups/0\fR\&\fR\&)\&. Execution properties for groups may also be specified, both for a top level group and for any of its sub-groups\&. Group execution properties specified here, will override properties in the group definition (see \fB\fIgroups/0\fR\&\fR\&)\&. (With value \fIdefault\fR\&, the group definition properties will be used)\&.
.LP
If \fI{skip,Reason}\fR\& is returned, all test cases in the module will be skipped, and the \fIReason\fR\& will be printed on the HTML result page\&.
.LP
For details on groups, see \fBTest case groups\fR\& in the User\&'s Guide\&.
.RE

.LP
.B
Module:groups() -> GroupDefs
.br
.RS
.LP
Types:

.RS 3
GroupDefs = [Group]
.br
Group = {GroupName,Properties,GroupsAndTestCases}
.br
GroupName = atom()
.br
Properties = [parallel | sequence | Shuffle | {RepeatType,N}]
.br
GroupsAndTestCases = [Group | {group,GroupName} | TestCase]
.br
TestCase = atom()
.br
Shuffle = shuffle | {shuffle,Seed}
.br
Seed = {integer(),integer(),integer()}
.br
RepeatType = repeat | repeat_until_all_ok | repeat_until_all_fail | repeat_until_any_ok | repeat_until_any_fail
.br
N = integer() | forever
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
Function for defining test case groups\&. Please see \fBTest case groups\fR\& in the User\&'s Guide for details\&.
.RE

.LP
.B
Module:suite() -> [Info] 
.br
.RS
.LP
Types:

.RS 3
 Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}
.br
 Time = TimeVal | TimeFunc
.br
 TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}
.br
 TimeFunc = {Mod,Func,Args} | Fun
.br
 MilliSec = integer()
.br
 Mod = atom()
.br
 Func = atom()
.br
 Args = list()
.br
 Fun = fun()
.br
 Required = Key | {Key,SubKeys} | {Key,SubKey} | {Key,SubKey,SubKeys}
.br
 Key = atom()
.br
 SubKeys = SubKey | [SubKey]
.br
 SubKey = atom()
.br
 Name = atom()
.br
 UserData = term()
.br
 Conns = [atom()]
.br
 CSSFile = string()
.br
 CTHs = [CTHModule |
.br
 {CTHModule, CTHInitArgs} |
.br
 {CTHModule, CTHInitArgs, CTHPriority}]
.br
 CTHModule = atom()
.br
 CTHInitArgs = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This is the test suite info function\&. It is supposed to return a list of tagged tuples that specify various properties related to the execution of this test suite (common for all test cases in the suite)\&.
.LP
The \fItimetrap\fR\& tag sets the maximum time each test case is allowed to execute (including \fB\fIinit_per_testcase/2\fR\&\fR\& and \fB\fIend_per_testcase/2\fR\&\fR\&)\&. If the timetrap time is exceeded, the test case fails with reason \fItimetrap_timeout\fR\&\&. A \fITimeFunc\fR\& function can be used to set a new timetrap by returning a \fITimeVal\fR\&\&. It may also be used to trigger a timetrap timeout by, at some point, returning a value other than a \fITimeVal\fR\&\&. (See the \fBUser\&'s Guide\fR\& for details)\&.
.LP
The \fIrequire\fR\& tag specifies configuration variables that are required by test cases (and/or configuration functions) in the suite\&. If the required configuration variables are not found in any of the configuration files, all test cases are skipped\&. For more information about the \&'require\&' functionality, see the reference manual for the function \fB\fIct:require/1/2\fR\&\fR\&\&.
.LP
With \fIuserdata\fR\&, it is possible for the user to specify arbitrary test suite related information which can be read by calling \fB\fIct:userdata/2\fR\&\fR\&\&.
.LP
The \fIct_hooks\fR\& tag specifies which \fBCommon Test Hooks\fR\& are to be run together with this suite\&.
.LP
Other tuples than the ones defined will simply be ignored\&.
.LP
For more information about the test suite info function, see \fBTest suite info function\fR\& in the User\&'s Guide\&.
.RE

.LP
.B
Module:init_per_suite(Config) -> NewConfig | {skip,Reason} | {skip_and_save,Reason,SaveConfig}
.br
.RS
.LP
Types:

.RS 3
 Config = NewConfig = SaveConfig = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
 Reason = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This configuration function is called as the first function in the suite\&. It typically contains initializations which are common for all test cases in the suite, and which shall only be done once\&. The \fIConfig\fR\& parameter is the configuration data which can be modified here\&. Whatever is returned from this function is given as \fIConfig\fR\& to all configuration functions and test cases in the suite\&. If \fI{skip,Reason}\fR\& is returned, all test cases in the suite will be skipped and \fIReason\fR\& printed in the overview log for the suite\&.
.LP
For information on \fIsave_config\fR\& and \fIskip_and_save\fR\&, please see \fBDependencies between Test Cases and Suites\fR\& in the User\&'s Guide\&.
.RE

.LP
.B
Module:end_per_suite(Config) -> void() | {save_config,SaveConfig}
.br
.RS
.LP
Types:

.RS 3
 Config = SaveConfig = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This function is called as the last test case in the suite\&. It is meant to be used for cleaning up after \fB\fIinit_per_suite/1\fR\&\fR\&\&. For information on \fIsave_config\fR\&, please see \fBDependencies between Test Cases and Suites\fR\& in the User\&'s Guide\&.
.RE

.LP
.B
Module:group(GroupName) -> [Info] 
.br
.RS
.LP
Types:

.RS 3
 Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns} | {stylesheet,CSSFile} | {ct_hooks, CTHs}
.br
 Time = TimeVal | TimeFunc
.br
 TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}
.br
 TimeFunc = {Mod,Func,Args} | Fun
.br
 MilliSec = integer()
.br
 Mod = atom()
.br
 Func = atom()
.br
 Args = list()
.br
 Fun = fun()
.br
 Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}
.br
 Key = atom()
.br
 SubKeys = SubKey | [SubKey]
.br
 SubKey = atom()
.br
 Name = atom()
.br
 UserData = term()
.br
 Conns = [atom()]
.br
 CSSFile = string()
.br
 CTHs = [CTHModule |
.br
 {CTHModule, CTHInitArgs} |
.br
 {CTHModule, CTHInitArgs, CTHPriority}]
.br
 CTHModule = atom()
.br
 CTHInitArgs = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This is the test case group info function\&. It is supposed to return a list of tagged tuples that specify various properties related to the execution of a test case group (i\&.e\&. its test cases and sub-groups)\&. Properties set by \fB\fIgroup/1\fR\&\fR\& override properties with the same key that have been previously set by \fB\fIsuite/0\fR\&\fR\&\&.
.LP
The \fItimetrap\fR\& tag sets the maximum time each test case is allowed to execute (including \fB\fIinit_per_testcase/2\fR\&\fR\& and \fB\fIend_per_testcase/2\fR\&\fR\&)\&. If the timetrap time is exceeded, the test case fails with reason \fItimetrap_timeout\fR\&\&. A \fITimeFunc\fR\& function can be used to set a new timetrap by returning a \fITimeVal\fR\&\&. It may also be used to trigger a timetrap timeout by, at some point, returning a value other than a \fITimeVal\fR\&\&. (See the \fBUser\&'s Guide\fR\& for details)\&.
.LP
The \fIrequire\fR\& tag specifies configuration variables that are required by test cases (and/or configuration functions) in the suite\&. If the required configuration variables are not found in any of the configuration files, all test cases in this group are skipped\&. For more information about the \&'require\&' functionality, see the reference manual for the function \fB\fIct:require/1/2\fR\&\fR\&\&.
.LP
With \fIuserdata\fR\&, it is possible for the user to specify arbitrary test case group related information which can be read by calling \fB\fIct:userdata/2\fR\&\fR\&\&.
.LP
The \fIct_hooks\fR\& tag specifies which \fBCommon Test Hooks\fR\& are to be run together with this suite\&.
.LP
Other tuples than the ones defined will simply be ignored\&.
.LP
For more information about the test case group info function, see \fBTest case group info function\fR\& in the User\&'s Guide\&.
.RE

.LP
.B
Module:init_per_group(GroupName, Config) -> NewConfig | {skip,Reason}
.br
.RS
.LP
Types:

.RS 3
 GroupName = atom()
.br
 Config = NewConfig = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
 Reason = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This configuration function is called before execution of a test case group\&. It typically contains initializations which are common for all test cases and sub-groups in the group, and which shall only be performed once\&. \fIGroupName\fR\& is the name of the group, as specified in the group definition (see \fB\fIgroups/0\fR\&\fR\&)\&. The \fIConfig\fR\& parameter is the configuration data which can be modified here\&. The return value of this function is given as \fIConfig\fR\& to all test cases and sub-groups in the group\&. If \fI{skip,Reason}\fR\& is returned, all test cases in the group will be skipped and \fIReason\fR\& printed in the overview log for the group\&.
.LP
For information about test case groups, please see \fBTest case groups\fR\& chapter in the User\&'s Guide\&.
.RE

.LP
.B
Module:end_per_group(GroupName, Config) -> void() | {return_group_result,Status}
.br
.RS
.LP
Types:

.RS 3
 GroupName = atom()
.br
 Config = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
 Status = ok | skipped | failed
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This function is called after the execution of a test case group is finished\&. It is meant to be used for cleaning up after \fB\fIinit_per_group/2\fR\&\fR\&\&. By means of \fI{return_group_result,Status}\fR\&, it is possible to return a status value for a nested sub-group\&. The status can be retrieved in \fB\fIend_per_group/2\fR\&\fR\& for the group on the level above\&. The status will also be used by Common Test for deciding if execution of a group should proceed in case the property \fIsequence\fR\& or \fIrepeat_until_*\fR\& is set\&.
.LP
For more information about test case groups, please see \fBTest case groups\fR\& chapter in the User\&'s Guide\&.
.RE

.LP
.B
Module:init_per_testcase(TestCase, Config) -> NewConfig | {fail,Reason} | {skip,Reason}
.br
.RS
.LP
Types:

.RS 3
 TestCase = atom()
.br
 Config = NewConfig = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
 Reason = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This function is called before each test case\&. The \fITestCase\fR\& argument is the name of the test case, and \fIConfig\fR\& (list of key-value tuples) is the configuration data that can be modified here\&. The \fINewConfig\fR\& list returned from this function is given as \fIConfig\fR\& to the test case\&. If \fI{fail,Reason}\fR\& is returned, the test case is marked as failed without being executed\&. If \fI{skip,Reason}\fR\& is returned, the test case will be skipped and \fIReason\fR\& printed in the overview log for the suite\&.
.RE

.LP
.B
Module:end_per_testcase(TestCase, Config) -> void() | {fail,Reason} | {save_config,SaveConfig}
.br
.RS
.LP
Types:

.RS 3
 TestCase = atom()
.br
 Config = SaveConfig = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
 Reason = term()
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This function is called after each test case, and can be used to clean up after \fB\fIinit_per_testcase/2\fR\&\fR\& and the test case\&. Any return value (besides \fI{fail,Reason}\fR\& and \fI{save_config,SaveConfig}\fR\&) is ignored\&. By returning \fI{fail,Reason}\fR\&, \fITestCase\fR\& will be marked as failed (even though it was actually successful in the sense that it returned a value instead of terminating)\&. For information on \fIsave_config\fR\&, please see \fBDependencies between Test Cases and Suites\fR\& in the User\&'s Guide
.RE

.LP
.B
Module:Testcase() -> [Info] 
.br
.RS
.LP
Types:

.RS 3
 Info = {timetrap,Time} | {require,Required} | {require,Name,Required} | {userdata,UserData} | {silent_connections,Conns}
.br
 Time = TimeVal | TimeFunc
.br
 TimeVal = MilliSec | {seconds,integer()} | {minutes,integer()} | {hours,integer()}
.br
 TimeFunc = {Mod,Func,Args} | Fun
.br
 MilliSec = integer()
.br
 Mod = atom()
.br
 Func = atom()
.br
 Args = list()
.br
 Fun = fun()
.br
 Required = Key | {Key,SubKeys} | {Key,Subkey} | {Key,Subkey,SubKeys}
.br
 Key = atom()
.br
 SubKeys = SubKey | [SubKey]
.br
 SubKey = atom()
.br
 Name = atom()
.br
 UserData = term()
.br
 Conns = [atom()]
.br
.RE
.RE
.RS
.LP
OPTIONAL
.LP
This is the test case info function\&. It is supposed to return a list of tagged tuples that specify various properties related to the execution of this particular test case\&. Properties set by \fB\fITestcase/0\fR\&\fR\& override properties that have been previously set for the test case by \fB\fIgroup/1\fR\&\fR\& or \fB\fIsuite/0\fR\&\fR\&\&.
.LP
The \fItimetrap\fR\& tag sets the maximum time the test case is allowed to execute\&. If the timetrap time is exceeded, the test case fails with reason \fItimetrap_timeout\fR\&\&. \fB\fIinit_per_testcase/2\fR\&\fR\& and \fB\fIend_per_testcase/2\fR\&\fR\& are included in the timetrap time\&. A \fITimeFunc\fR\& function can be used to set a new timetrap by returning a \fITimeVal\fR\&\&. It may also be used to trigger a timetrap timeout by, at some point, returning a value other than a \fITimeVal\fR\&\&. (See the \fBUser\&'s Guide\fR\& for details)\&.
.LP
The \fIrequire\fR\& tag specifies configuration variables that are required by the test case (and/or \fIinit/end_per_testcase/2\fR\&)\&. If the required configuration variables are not found in any of the configuration files, the test case is skipped\&. For more information about the \&'require\&' functionality, see the reference manual for the function \fB\fIct:require/1/2\fR\&\fR\&\&.
.LP
If \fItimetrap\fR\& and/or \fIrequire\fR\& is not set, the default values specified by \fB\fIsuite/0\fR\&\fR\& (or \fB\fIgroup/1\fR\&\fR\&) will be used\&.
.LP
With \fIuserdata\fR\&, it is possible for the user to specify arbitrary test case related information which can be read by calling \fB\fIct:userdata/3\fR\&\fR\&\&.
.LP
Other tuples than the ones defined will simply be ignored\&.
.LP
For more information about the test case info function, see \fBTest case info function\fR\& in the User\&'s Guide\&.
.RE

.LP
.B
Module:Testcase(Config) -> void() | {skip,Reason} | {comment,Comment} | {save_config,SaveConfig} | {skip_and_save,Reason,SaveConfig} | exit() 
.br
.RS
.LP
Types:

.RS 3
 Config = SaveConfig = [{Key,Value}]
.br
 Key = atom()
.br
 Value = term()
.br
 Reason = term()
.br
 Comment = string()
.br
.RE
.RE
.RS
.LP
MANDATORY
.LP
This is the implementation of a test case\&. Here you must call the functions you want to test, and do whatever you need to check the result\&. If something fails, make sure the function causes a runtime error, or call \fB\fIct:fail/1/2\fR\&\fR\& (which also causes the test case process to terminate)\&.
.LP
Elements from the \fIConfig\fR\& list can e\&.g\&. be read with \fIproplists:get_value/2\fR\& (or the macro \fI?config\fR\& defined in \fIct\&.hrl\fR\&)\&.
.LP
You can return \fI{skip,Reason}\fR\& if you decide not to run the test case after all\&. \fIReason\fR\& will then be printed in \&'Comment\&' field on the HTML result page\&.
.LP
You can return \fI{comment,Comment}\fR\& if you wish to print some information in the \&'Comment\&' field on the HTML result page\&.
.LP
If the function returns anything else, the test case is considered successful\&. (The return value always gets printed in the test case log file)\&.
.LP
For more information about test case implementation, please see \fBTest cases\fR\& in the User\&'s Guide\&.
.LP
For information on \fIsave_config\fR\& and \fIskip_and_save\fR\&, please see \fBDependencies between Test Cases and Suites\fR\& in the User\&'s Guide\&.
.RE