NAME¶
aegis new test - add a new test to a change
SYNOPSIS¶
aegis -New_Test [
option... ][
filename... ]
aegis -New_Test -List [
option... ]
aegis -New_Test -Help
DESCRIPTION¶
The
aegis -New_Test command is used to add a new test to a change.
A new file is created in the development directory.
New tests default to “automatic” unless otherwise specified.
File Name Interpretation¶
The aegis program will attempt to determine the project file names from the file
names given on the command line. All file names are stored within aegis
projects as relative to the root of the baseline directory tree. The
development directory and the integration directory are shadows of this
baseline directory, and so these relative names apply here, too. Files named
on the command line are first converted to absolute paths if necessary. They
are then compared with the baseline path, the development directory path, and
the integration directory path, to determine a baseline-relative name. It is
an error if the file named is outside one of these directory trees.
The
-BAse_RElative option may be used to cause relative filenames to be
interpreted as relative to the baseline path; absolute filenames will still be
compared with the various paths in order to determine a baseline-relative
name.
The
relative_filename_preference in the user configuration file may be
used to modify this default behavior. See
aeuconf(5) for more
information.
Test Filename Generation¶
You may choose your own filename for a test, by specifying it on the command
line.
If no filename is specified on the command line, a test filename is
automatically generated. This is controlled by the
new_test_filename
field of the project configuration file (see
aepconf(5) for more
information. All automatically generated test filenames within a project are
numbered uniquely. The default pattern for new test filenames is "
test/XX/tXXXX[am].sh", where
XX is the first 2 digits of
the test number,
XXXX is the whole test number, and
[am] is a
for automatic tests and m for manual tests.
Modifying Tests¶
Tests may be modified in future by adding them to a change with the
aecp(1) command. Tests are treated just like any other source file, and
are subject to the same process.
File Templates¶
When a new file is created in the development directory the project
config file is searched for a template for the new file. If a template
is found, the new file will be initialized to the template, otherwise it will
be created empty. See
aepconf(5) for more information.
The simplest form is to use template files, such as
file_template =
[
{
pattern = [ "*.c" ];
body = "${read_file ${source template/c abs}}";
},
{
pattern = [ "test/*/.sh" ];
body = "${read_file ${source template/test abs}}";
},
];
As you can see, the template files are part of the project source, so you can
add the appropriate copyright notices, and wrappers,
etc. The
$source substitution locates them, if they are not part of the current
change (and they usually are not).
The template files themselves contain substitutions. The
$filename
substitution is available, and contains the name of the file being created.
This can be manipulated in various ways when constructing the appropriate file
contents. See
aesub(5) for more information about substitutions.
It is also possible to run a command to create the new file. You can do this
instead of specifying a body string,
viz:
file_template =
[
{
pattern = [ "*" ];
body_command = "perl ${source template.pl abs} $filename";
},
];
The command is run with a current directory set to the top of the development
directory. It is an error if the command fails to create the file. You can
mix-and-match the two techniques,
body string and
body_command,
if you want.
Be careful to make sure that the test filename template pattern matches the
new_test_filename field.
File Name Limitations¶
There are a number of controls available to limit the form of project file
names. All of these controls may be found in the project configuration file,
see
aepconf(5) for more information. The most significant are briefly
described here:
- maximum_filename_length = integer;
- This field is used to limit the length of filenames. All
new files may not have path components longer than this. Defaults to 255
if not set. For maximum portability you should set this to 14.
- posix_filename_charset = boolean;
- This field may be used to limit the characters allowed in
filenames to only those explicitly allowed by POSIX. Defaults to
false if not set, meaning whatever your operating system will
tolerate, except white space and high-bit-on characters. For maximum
portability you should set this to true.
- dos_filename_required = boolean;
- This field may be used to limit filenames so that they
conform to the DOS 8+3 filename limits and to the DOS filename character
set. Defaults to false if not set.
- windows_filename_required = boolean;
- This field may be used to limit filenames so that they
conform to the Windows98 and WindowsNT filename limits and character set.
Defaults to false if not set.
- shell_safe_filenames = boolean;
- This field may be used to limit filenames so that they do
not contain shell special characters. Defaults to true if not set.
If this field is set to false, you will need to use the
${quote} substitution around filenames in commands, to ensure that
filenames containing shell special characters do not have unintended side
effects. Weird characters in filenames may also confuse your dependency
maintenance tool.
- allow_white_space_in_filenames = boolean;
This field may be used to allow white space
characters in file names. This will allow the following characters to appear
in file names: backspace (BS, \b, 0x08), horizontal tab (HT, \t, 0x09), new
line (NL, \n, 0x0A), vertical tab (VT, \v, 0x0B), form feed (FF, \f, 0x0C),
and carriage return (CR, \r, 0x0D). Defaults to false if not set.
Note that this field does not override other file name filters. It will be
necessary to explicitly set
shell_safe_filenames = false as well. It
will be necessary to set
dos_filename_required = false (the default) as
well. It will be necessary to set
posix_filename_charset = false (the
default) as well.
The user must take great care to use the ${quote} substitution around all file
names in commands in the project configuration. And even then, substitutions
which expect a space separated list of file names will have undefined
results.
- allow_non_ascii_filenames = boolean;
This field may be used to allow file names
with non-ascii-printable characters in them. Usually this would mean a UTF8 or
international charset of some kind. Defaults to false if not set.
Note that this field does not override other file name filters. It will be
necessary to explicitly set
shell_safe_filenames = false as well. It
will be necessary to set
dos_filename_required = false (the default) as
well. It will be necessary to set
posix_filename_charset = false (the
default) as well.
- filename_pattern_accept = [ string ];
- This field is used to specify a list of patterns of
acceptable filenames. Defaults to "*" if not set.
- filename_pattern_reject = [ string ];
-
This field is used to specify a list of patterns of unacceptable
filenames.
Please Note: Aegis also consults the underlying file system, to determine
its notion of maximum file size. Where the file system's maximum file size is
less than
maximum_filename_length, the filesystem wins. This can
happen, for example, when you are using the Linux UMSDOS file system, or when
you have an NFS mounted an ancient V7 filesystem. Setting
maximum_filename_length to 255 in these cases does not alter the fact
that the underlying file systems limits are far smaller (12 and 14,
respectively).
If your development directories (or your whole project) is on filesystems with
filename limitations, or a portion of the heterogeneous builds take place in
such an environment, it helps to tell Aegis what they are (using the project
config file's fields) so that you don't run into the situation where
the project builds on the more permissive environments, but fails with
mysterious errors in the more limited environments.
If your development directories are routinely on a Linux UMSDOS filesystem, you
would probably be better off setting
dos_filename_required = true, and
also changing the
development_directory_template field. Heterogeneous
development with various Windows environments may also require this.
Changing the Type of a File¶
If you want to change the type of a file (say, from a test to a source file, or
vice versa) you could do it as two changes, by first using
aerm(1) in one change and then using
aenf(1) or
aent(1)
in a second change, or you can combine both steps in the same change. Remember
to use the
aerm -nowhiteout option or you will get a most peculiar new
file template.
Notification¶
The
new_test_command in the project
config file is run, if set.
The
project_file_command is also run, if set, and if there has been an
integration recently. See
aepconf(5) for more information.
TEST PROCESS¶
Each change is required to be accompanied by tests, and those tests are required
to be run against the built development directory, and they must pass. This
ensures that new functionality is accompanied by tests to verify its
correctness, and bug fixes are accompanied by tests which confirm that the bug
has been fixed.
Regression Tests¶
Tests are treated as any other source file, and are maintained in the baseline
and history with all other source files. The tests which must accompany every
change accumulate in the project baseline, providing a definition of correct
function for the baseline. These accumulated tests may be executed using an
“aegis -REGression” command, to verify that the project will not
“regress” as a result of a change.
Baseline Tests¶
Bug fixes are required to have their tests
fail against the project
baseline (in contrast to the development directory). This ensures that the
test actually demonstrates the bug in the baseline, as well as demonstrating
that it is fixed by the change. New functionality trivially fails against the
baseline, and so aegis does not attempt to guess if a test is a bug fix test
or new functionality test, it simply requires tests to fail against the
baseline.
This requirement applies both to new tests being created by a change and also to
tests which have been copied into a change for modification.
Reviewing Tests¶
Reviewers may be confident that aegis has enforced the test requirements; that a
change must have tests, that the change must build, that the tests pass
against the development directory, and that the tests fail against the
baseline. These conditions are enforced by
aede(1) and the change will
not be advanced to the
being reviewed state until these conditions are
met. Reviewers should thus review tests for
completeness of coverage of
the code in the change, and insensitivity to changes in the execution
environment (e.g. not date sensitive). Reviewers should also use “aegis
-list change_details” to verify that a change does or does not have
testing exemptions.
Exemptions¶
Various test exemptions may be granted by project administrators, see
aepa(1) and
aepattr(5) for more information. Copying tests into
a change, or adding new tests to a change, may cancel those exemptions.
TEST CORRELATIONS¶
The “aegis -Test -SUGgest” command may be used to have aegis suggest
suitable regression tests for your change, based on the source files in your
change. This automatically focuses testing effort to relevant tests, reducing
the number of regression tests necessary to be confident that you have not
introduced a bug.
The test correlations are generated by the “aegis -Integrate_Pass”
command, which associates each test in the change with each source file in the
change. Thus, each source file accumulates a list of tests which have been
associated with it in the past. This is not as exact as code coverage
analysis, but is a reasonable approximation in practice.
The
aecp(1) and
aenf(1) commands are used to associate files with
a change. While they do not actively perform the association, these are the
files used by
aeipass(1) and
aet(1) to determine which source
files are associated with which tests.
Test Correlation Accuracy¶
Assuming that the testing correlations are accurate and that the tests are
evenly distributed across the function space, there will be a less than
1/number chance that a relevant test has not been run by the
“aegis -Test -SUGgest
number” command. A small amount of
noise is added to the test weighting, so that unexpected things are sometimes
tested, and the same tests are not run every time.
Test correlation accuracy can be improved by ensuring that:
- •
- Each change should be strongly focused, with no gratuitous
file inclusions. This avoids spurious correlations.
- •
- Each item of new functionality should be added in an
individual change, rather than several together. This strongly correlates
tests with functionality.
- •
- Each bug should be fixed in an individual change, rather
than several together. This strongly correlates tests with
functionality.
- •
- Test correlations will be lost if files are moved. This is
because correlations are by name.
The best way for tests to correlate accurately with source files is when a
change contains a test and exactly those files relating to the functionality
under test. Too many spurious files will weaken the usefulness of the testing
correlations.
OPTIONS¶
The following options are understood;
- -AUTOmatic
- This option may be used to specify automatic tests.
Automatic tests require no human assistance.
- -BAse_RElative
- This option may be used to cause relative filenames to be
considered relative to the base of the source tree. See aeuconf(5)
for the corresponding user preference.
- -CUrrent_RElative
- This option may be used to cause relative filenames to be
considered relative to the current directory. This is usually the default.
See aeuconf(5) for the corresponding user preference.
- -Change number
- This option may be used to specify a particular change
within a project. See aegis(1) for a complete description of this
option.
- -Help
-
This option may be used to obtain more information about how to use the
aegis program.
- -List
-
This option may be used to obtain a list of suitable subjects for this
command. The list may be more general than expected.
- -MANual
- This option may be used to specify manual tests. Manual
tests require some human intervention, e.g.: confirmation of some screen
behavior (X11, for instance), or some user action, "unplug ethernet
cable now".
- -Not_Logging
-
This option may be used to disable the automatic logging of output and
errors to a file. This is often useful when several aegis commands are
combined in a shell script.
- -Output filename
- This option may be used to specify a filename which is to
be written with the automatically determined test file name. Useful for
writing scripts.
- -Project name
- This option may be used to select the project of interest.
When no -Project option is specified, the AEGIS_PROJECT
environment variable is consulted. If that does not exist, the user's
$HOME/.aegisrc file is examined for a default project field (see
aeuconf(5) for more information). If that does not exist, when the
user is only working on changes within a single project, the project name
defaults to that project. Otherwise, it is an error.
- -TEMplate
- This option may be used to specify that a new file template
should be used, even if the file already exists.
- -No_TEMplate
- This option may be used to specify that a new file template
should not be used, even if the file does not exist (any empty file will
be created).
- -TERse
-
This option may be used to cause listings to produce the bare minimum of
information. It is usually useful for shell scripts.
- -Verbose
- This option may be used to cause aegis to produce more
output. By default aegis only produces output on errors. When used with
the -List option this option causes column headings to be
added.
- -Wait
- This option may be used to require Aegis commands to wait
for access locks, if they cannot be obtained immediately. Defaults to the
user's lock_wait_preference if not specified, see aeuconf(5)
for more information.
- -No_Wait
- This option may be used to require Aegis commands to emit a
fatal error if access locks cannot be obtained immediately. Defaults to
the user's lock_wait_preference if not specified, see
aeuconf(5) for more information.
See also
aegis(1) for options common to all aegis commands.
All options may be abbreviated; the abbreviation is documented as the upper case
letters, all lower case letters and underscores (_) are optional. You must use
consecutive sequences of optional letters.
All options are case insensitive, you may type them in upper case or lower case
or a combination of both, case is not important.
For example: the arguments "-project, "-PROJ" and "-p"
are all interpreted to mean the
-Project option. The argument
"-prj" will not be understood, because consecutive optional
characters were not supplied.
Options and other command line arguments may be mixed arbitrarily on the command
line, after the function selectors.
The GNU long option names are understood. Since all option names for
aegis are long, this means ignoring the extra leading '-'. The "
--option=value" convention is also
understood.
RECOMMENDED ALIAS¶
The recommended alias for this command is
csh% alias aent 'aegis -nt \!* -v'
sh$ aent(){aegis -nt "$@" -v}
ERRORS¶
It is an error if the change is not in the
being developed state.
It is an error if the change is not assigned to the current user.
EXIT STATUS¶
The
aegis command will exit with a status of 1 on any error. The
aegis command will only exit with a status of 0 if there are no errors.
ENVIRONMENT VARIABLES¶
See
aegis(1) for a list of environment variables which may affect this
command. See
aepconf(5) for the project configuration file's
project_specific field for how to set environment variables for all
commands executed by Aegis.
SEE ALSO¶
- aecp(1)
- copy an existing test into a change
- aedb(1)
- begin development of a change
- aentu(1)
- remove a new test from a change
- aerm(1)
- remove an existing test as part of a change
- aet(1)
- run tests
- aeuconf(5)
- user configuration file format
COPYRIGHT¶
aegis version 4.24.3.D001
Copyright (C) 1991, 1992, 1993, 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001,
2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010 Peter Miller
The aegis program comes with ABSOLUTELY NO WARRANTY; for details use the '
aegis -VERSion License' command. This is free software and you are
welcome to redistribute it under certain conditions; for details use the '
aegis -VERSion License' command.
AUTHOR¶