other versions
- jessie 4.24.3-3
aegis -New_Test(1) | General Commands Manual | aegis -New_Test(1) |
NAME¶
aegis new test - add a new test to a changeSYNOPSIS¶
aegis -New_Test [ option... ][ filename... ]DESCRIPTION¶
The aegis -New_Test command is used to add a new test to a change. A new file is created in the development directory. New tests default to “automatic” unless otherwise specified.File Name Interpretation¶
The aegis program will attempt to determine the project file names from the file names given on the command line. All file names are stored within aegis projects as relative to the root of the baseline directory tree. The development directory and the integration directory are shadows of this baseline directory, and so these relative names apply here, too. Files named on the command line are first converted to absolute paths if necessary. They are then compared with the baseline path, the development directory path, and the integration directory path, to determine a baseline-relative name. It is an error if the file named is outside one of these directory trees. The -BAse_RElative option may be used to cause relative filenames to be interpreted as relative to the baseline path; absolute filenames will still be compared with the various paths in order to determine a baseline-relative name. The relative_filename_preference in the user configuration file may be used to modify this default behavior. See aeuconf(5) for more information.Test Filename Generation¶
You may choose your own filename for a test, by specifying it on the command line. If no filename is specified on the command line, a test filename is automatically generated. This is controlled by the new_test_filename field of the project configuration file (see aepconf(5) for more information. All automatically generated test filenames within a project are numbered uniquely. The default pattern for new test filenames is " test/XX/tXXXX[am].sh", where XX is the first 2 digits of the test number, XXXX is the whole test number, and [am] is a for automatic tests and m for manual tests.Modifying Tests¶
Tests may be modified in future by adding them to a change with the aecp(1) command. Tests are treated just like any other source file, and are subject to the same process.File Templates¶
When a new file is created in the development directory the project config file is searched for a template for the new file. If a template is found, the new file will be initialized to the template, otherwise it will be created empty. See aepconf(5) for more information. The simplest form is to use template files, such asfile_template = [ { pattern = [ "*.c" ]; body = "${read_file ${source template/c abs}}"; }, { pattern = [ "test/*/.sh" ]; body = "${read_file ${source template/test abs}}"; }, ];
file_template = [ { pattern = [ "*" ]; body_command = "perl ${source template.pl abs} $filename"; }, ];
File Name Limitations¶
There are a number of controls available to limit the form of project file names. All of these controls may be found in the project configuration file, see aepconf(5) for more information. The most significant are briefly described here:- maximum_filename_length = integer;
- This field is used to limit the length of filenames. All new files may not have path components longer than this. Defaults to 255 if not set. For maximum portability you should set this to 14.
- posix_filename_charset = boolean;
- This field may be used to limit the characters allowed in filenames to only those explicitly allowed by POSIX. Defaults to false if not set, meaning whatever your operating system will tolerate, except white space and high-bit-on characters. For maximum portability you should set this to true.
- dos_filename_required = boolean;
- This field may be used to limit filenames so that they conform to the DOS 8+3 filename limits and to the DOS filename character set. Defaults to false if not set.
- windows_filename_required = boolean;
- This field may be used to limit filenames so that they conform to the Windows98 and WindowsNT filename limits and character set. Defaults to false if not set.
- shell_safe_filenames = boolean;
- This field may be used to limit filenames so that they do not contain shell special characters. Defaults to true if not set. If this field is set to false, you will need to use the ${quote} substitution around filenames in commands, to ensure that filenames containing shell special characters do not have unintended side effects. Weird characters in filenames may also confuse your dependency maintenance tool.
- allow_white_space_in_filenames = boolean;
This field may be used to allow white space characters in
file names. This will allow the following characters to appear in file names:
backspace (BS, \b, 0x08), horizontal tab (HT, \t, 0x09), new line (NL, \n,
0x0A), vertical tab (VT, \v, 0x0B), form feed (FF, \f, 0x0C), and carriage
return (CR, \r, 0x0D). Defaults to false if not set.
Note that this field does not override other file name filters. It will be
necessary to explicitly set shell_safe_filenames = false as well. It
will be necessary to set dos_filename_required = false (the default) as
well. It will be necessary to set posix_filename_charset = false (the
default) as well.
The user must take great care to use the ${quote} substitution around all file
names in commands in the project configuration. And even then, substitutions
which expect a space separated list of file names will have undefined
results.
- allow_non_ascii_filenames = boolean;
This field may be used to allow file names with
non-ascii-printable characters in them. Usually this would mean a UTF8 or
international charset of some kind. Defaults to false if not set.
Note that this field does not override other file name filters. It will be
necessary to explicitly set shell_safe_filenames = false as well. It
will be necessary to set dos_filename_required = false (the default) as
well. It will be necessary to set posix_filename_charset = false (the
default) as well.
- filename_pattern_accept = [ string ];
- This field is used to specify a list of patterns of acceptable filenames. Defaults to "*" if not set.
- filename_pattern_reject = [ string ];
-
Changing the Type of a File¶
If you want to change the type of a file (say, from a test to a source file, or vice versa) you could do it as two changes, by first using aerm(1) in one change and then using aenf(1) or aent(1) in a second change, or you can combine both steps in the same change. Remember to use the aerm -nowhiteout option or you will get a most peculiar new file template.Notification¶
The new_test_command in the project config file is run, if set. The project_file_command is also run, if set, and if there has been an integration recently. See aepconf(5) for more information.TEST PROCESS¶
Each change is required to be accompanied by tests, and those tests are required to be run against the built development directory, and they must pass. This ensures that new functionality is accompanied by tests to verify its correctness, and bug fixes are accompanied by tests which confirm that the bug has been fixed.Regression Tests¶
Tests are treated as any other source file, and are maintained in the baseline and history with all other source files. The tests which must accompany every change accumulate in the project baseline, providing a definition of correct function for the baseline. These accumulated tests may be executed using an “aegis -REGression” command, to verify that the project will not “regress” as a result of a change.Baseline Tests¶
Bug fixes are required to have their tests fail against the project baseline (in contrast to the development directory). This ensures that the test actually demonstrates the bug in the baseline, as well as demonstrating that it is fixed by the change. New functionality trivially fails against the baseline, and so aegis does not attempt to guess if a test is a bug fix test or new functionality test, it simply requires tests to fail against the baseline. This requirement applies both to new tests being created by a change and also to tests which have been copied into a change for modification.Reviewing Tests¶
Reviewers may be confident that aegis has enforced the test requirements; that a change must have tests, that the change must build, that the tests pass against the development directory, and that the tests fail against the baseline. These conditions are enforced by aede(1) and the change will not be advanced to the being reviewed state until these conditions are met. Reviewers should thus review tests for completeness of coverage of the code in the change, and insensitivity to changes in the execution environment (e.g. not date sensitive). Reviewers should also use “aegis -list change_details” to verify that a change does or does not have testing exemptions.Exemptions¶
Various test exemptions may be granted by project administrators, see aepa(1) and aepattr(5) for more information. Copying tests into a change, or adding new tests to a change, may cancel those exemptions.TEST CORRELATIONS¶
The “aegis -Test -SUGgest” command may be used to have aegis suggest suitable regression tests for your change, based on the source files in your change. This automatically focuses testing effort to relevant tests, reducing the number of regression tests necessary to be confident that you have not introduced a bug. The test correlations are generated by the “aegis -Integrate_Pass” command, which associates each test in the change with each source file in the change. Thus, each source file accumulates a list of tests which have been associated with it in the past. This is not as exact as code coverage analysis, but is a reasonable approximation in practice. The aecp(1) and aenf(1) commands are used to associate files with a change. While they do not actively perform the association, these are the files used by aeipass(1) and aet(1) to determine which source files are associated with which tests.Test Correlation Accuracy¶
Assuming that the testing correlations are accurate and that the tests are evenly distributed across the function space, there will be a less than 1/number chance that a relevant test has not been run by the “aegis -Test -SUGgest number” command. A small amount of noise is added to the test weighting, so that unexpected things are sometimes tested, and the same tests are not run every time. Test correlation accuracy can be improved by ensuring that:- •
- Each change should be strongly focused, with no gratuitous file inclusions. This avoids spurious correlations.
- •
- Each item of new functionality should be added in an individual change, rather than several together. This strongly correlates tests with functionality.
- •
- Each bug should be fixed in an individual change, rather than several together. This strongly correlates tests with functionality.
- •
- Test correlations will be lost if files are moved. This is because correlations are by name.
OPTIONS¶
The following options are understood;- -AUTOmatic
- This option may be used to specify automatic tests. Automatic tests require no human assistance.
- -BAse_RElative
- This option may be used to cause relative filenames to be considered relative to the base of the source tree. See aeuconf(5) for the corresponding user preference.
- -CUrrent_RElative
- This option may be used to cause relative filenames to be considered relative to the current directory. This is usually the default. See aeuconf(5) for the corresponding user preference.
- -Change number
- This option may be used to specify a particular change within a project. See aegis(1) for a complete description of this option.
- -Help
-
- -List
-
- -MANual
- This option may be used to specify manual tests. Manual tests require some human intervention, e.g.: confirmation of some screen behavior (X11, for instance), or some user action, "unplug ethernet cable now".
- -Not_Logging
-
- -Output filename
- This option may be used to specify a filename which is to be written with the automatically determined test file name. Useful for writing scripts.
- -Project name
- This option may be used to select the project of interest. When no -Project option is specified, the AEGIS_PROJECT environment variable is consulted. If that does not exist, the user's $HOME/.aegisrc file is examined for a default project field (see aeuconf(5) for more information). If that does not exist, when the user is only working on changes within a single project, the project name defaults to that project. Otherwise, it is an error.
- -TEMplate
- This option may be used to specify that a new file template should be used, even if the file already exists.
- -No_TEMplate
- This option may be used to specify that a new file template should not be used, even if the file does not exist (any empty file will be created).
- -TERse
-
- -Verbose
- This option may be used to cause aegis to produce more output. By default aegis only produces output on errors. When used with the -List option this option causes column headings to be added.
- -Wait
- This option may be used to require Aegis commands to wait for access locks, if they cannot be obtained immediately. Defaults to the user's lock_wait_preference if not specified, see aeuconf(5) for more information.
- -No_Wait
- This option may be used to require Aegis commands to emit a fatal error if access locks cannot be obtained immediately. Defaults to the user's lock_wait_preference if not specified, see aeuconf(5) for more information.
RECOMMENDED ALIAS¶
The recommended alias for this command iscsh% alias aent 'aegis -nt \!* -v' sh$ aent(){aegis -nt "$@" -v}
ERRORS¶
It is an error if the change is not in the being developed state.EXIT STATUS¶
The aegis command will exit with a status of 1 on any error. The aegis command will only exit with a status of 0 if there are no errors.ENVIRONMENT VARIABLES¶
See aegis(1) for a list of environment variables which may affect this command. See aepconf(5) for the project configuration file's project_specific field for how to set environment variables for all commands executed by Aegis.SEE ALSO¶
- aecp(1)
- copy an existing test into a change
- aedb(1)
- begin development of a change
- aentu(1)
- remove a new test from a change
- aerm(1)
- remove an existing test as part of a change
- aet(1)
- run tests
- aeuconf(5)
- user configuration file format
COPYRIGHT¶
aegis version 4.24.3.D001AUTHOR¶
Peter Miller | E-Mail: | millerp@canb.auug.org.au |
/\/\* | WWW: | http://www.canb.auug.org.au/~millerp/ |
Aegis | Reference Manual |