NAME¶
CPAN::Reporter::Smoker - Turnkey CPAN Testers smoking
VERSION¶
version 0.25
SYNOPSIS¶
$ perl -MCPAN::Reporter::Smoker -e start
DESCRIPTION¶
Rudimentary smoke tester for CPAN Testers, built upon CPAN::Reporter. Use at
your own risk. It requires a recent version of CPAN::Reporter to run.
Currently, CPAN::Reporter::Smoker requires zero independent configuration;
instead it uses configuration settings from CPAN.pm and CPAN::Reporter.
Once started, it retrieves a list of distributions from the configured CPAN
mirror and begins testing them in reverse order of upload. It will skip any
distribution which has already had a report sent by CPAN::Reporter.
Features (or bugs, depending on your point of view):
- •
- No configuration needed
- •
- Tests each distribution as a separate CPAN process -- each distribution
has prerequisites like build_requires satisfied from scratch
- •
- Automatically checks for new distributions every twelve hours or as
otherwise specified
- •
- Continues until interrupted with CTRL-C
- •
- Checks CPAN.pm "distroprefs" to see if distributions should be
skipped (before handing off to CPAN)
Current limitations:
- •
- Does not attempt to retest distributions that had reports discarded
because of prerequisites that could not be satisfied
WARNING -- smoke testing is risky¶
Smoke testing will download and run programs that other people have uploaded to
CPAN. These programs could do
anything to your system, including
deleting everything on it. Do not run CPAN::Reporter::Smoker unless you are
prepared to take these risks.
USAGE¶
"start()"¶
Starts smoke testing using defaults already in CPAN::Config and CPAN::Reporter's
.cpanreporter directory. Runs until all distributions are tested or the
process is halted with CTRL-C or otherwise killed.
"start()" supports several optional arguments:
- •
- "clean_cache_after" -- number of distributions that will be
tested before checking to see if the CPAN build cache needs to be cleaned
up (not including any prerequisites tested). Must be a positive integer.
Defaults to 100
- •
- "list" -- if provided, this list of distributions will be tested
instead of all of CPAN. May be a reference to an array of distribution
names or may be a filename containing one distribution name per line.
Distribution names must be of the form 'AUTHOR/Dist-Name-0.00.tar.gz'
- •
- "restart_delay" -- number of seconds that must elapse before
restarting smoke testing. This will reload indices to search for new
distributions and restart testing from the most recent distribution. Must
be a positive integer; Defaults to 43200 seconds (12 hours)
- •
- "set_term_title" -- toggle for whether the terminal titlebar
will be updated with the distribution being smoke tested and the starting
time of the test. Helps determine if a test is hung and which distribution
might be responsible. Valid values are 0 or 1. Defaults to 1
- •
- "status_file" -- during testing, the name of the distribution
under test and a timestamp are written to this file. The file is removed
after the test is complete. This helps identify a problem distribution if
testing hangs or crashes the computer. If the argument includes a path,
all directories to the file must exist. Defaults to
"smoker-status-$$.txt" in File::Spec->tmpdir.
- •
- "install" -- toggle for whether the distribution should be
installed after successful testing. Can be useful to avoid prerequisite
re-building and growing PERL5LIB for the cost of disk space used for
installed modules. Valid values are 0 or 1. Defaults to 0
- •
- "reverse" -- toggle the order in which releases are tested. When
set to 1, testing starts from the older release not the most recent one
(or the last distribution if --list is provided). Valid values are 0 or 1.
Defaults to 0
- •
- "force_trust" -- toggle whether to override CPAN's
"trust_test_report_history" option. When set to 1,
"trust_test_report_history" is set to 1. When set to 0,
"trust_test_report_history" is left alone and whatever the user
has configured for their CPAN client is used. Valid values are 0 or 1.
Defaults to 0
- •
- "reload_history_period" -- after this period in seconds, history
of modules smoked will be reloaded when possible. Default value 1800
seconds (30 minutes).
HINTS¶
Selection of distributions to test¶
Only the most recently uploaded developer and normal releases will be tested,
and only if the developer release is newer than the regular release indexed by
PAUSE.
For example, if Foo-Bar-0.01, Foo-Bar-0.02, Foo-Bar-0.03_01 and Foo-Bar-0.03_02
are on CPAN, only Foo-Bar-0.02 and Foo-Bar-0.03_02 will be tested, and in
reverse order of when they were uploaded. Once Foo-Bar-0.04 is released and
indexed, Foo-Bar-0.03_02 will not longer be tested.
To avoid testing script or other tarballs, developer distributions included must
have a base distribution name that resembles a distribution tarball already
indexed by PAUSE. If the first upload of distribution to PAUSE is a developer
release -- Baz-Bam-0.00_01.tar.gz -- it will not be tested as there is no
indexed Baz-Bam appearing in CPAN's 02packages.details.txt file.
Unauthorized tarballs are treated like developer releases and will be tested if
they resemble an indexed distribution and are newer than the indexed tarball.
Perl, parrot, kurila, Pugs and similar distributions will not be tested. The
skip list is based on CPAN::Mini and matches as follows:
qr{(?:
/(?:emb|syb|bio)?perl-\d
| /(?:parrot|ponie|kurila|Perl6-Pugs)-\d
| /perl-?5\.004
| /perl_mlb\.zip
)}xi,
Bundles and mod_perl distributions will also not be tested, though mod_perl is
likely to be requested as a dependency by many modules. See the next section
for how to tell CPAN.pm not to test certain dependencies.
Skipping additional distributions¶
If certain distributions hang, crash or otherwise cause trouble, you can use
CPAN's "distroprefs" system to disable them. If a distribution is
disabled, it won't be built or tested. If a distribution's dependency is
disabled, a failing test is just discarded.
The first step is configuring a directory for distroprefs files:
$ cpan
cpan> o conf init prefs_dir
cpan> o conf commit
Next, ensure that either the YAML or YAML::Syck module is installed. (YAML::Syck
is faster). Then create a file in the "prefs_dir" directory to hold
the list of distributions to disable, e.g. call it "disabled.yml"
In that file, you can add blocks of YAML code to disable distributions. The
match criteria "distribution" is a regex that matches against the
canonical name of a distribution, e.g. "AUTHOR/Foo-Bar-3.14.tar.gz".
Here is a sample file to show you some syntax (don't actually use these,
though):
---
comment: "Tests take too long"
match:
distribution: "^DAGOLDEN/CPAN-Reporter-\d"
disabled: 1
---
comment: "Skip Win32 distributions"
match:
distribution: "/Win32"
disabled: 1
---
comment: "Skip distributions by Andy Lester"
match:
distribution: "^PETDANCE"
disabled: 1
Please note that disabling distributions like this will also disable them for
normal, non-smoke usage of CPAN.pm.
One distribution that I would recommend either installing up front or else
disabling with distroprefs is mod_perl, as it is a common requirement for many
Apache:: modules but does not (easily) build and test under automation.
---
comment: "Don't build mod_perl if required by some other module"
match:
distribution: "/mod_perl-\d"
disabled: 1
Distroprefs are more powerful than this -- they can be used to automate
responses to prompts in distributions, set environment variables, specify
additional dependencies and so on. Read the docs for CPAN.pm for more and look
in the "distroprefs" directory in the CPAN distribution tarball for
examples.
Using a local CPAN::Mini mirror¶
Because distributions must be retrieved from a CPAN mirror, the smoker may cause
heavy network load and will repetitively download common build prerequisites.
An alternative is to use CPAN::Mini to create a local CPAN mirror and to point
CPAN's "urllist" to the local mirror.
$ cpan
cpan> o conf urllist unshift file:///path/to/minicpan
cpan> o conf commit
However, CPAN::Reporter::Smoker needs the "find-ls.gz" file, which
CPAN::Mini does not mirror by default. Add it to a .minicpanrc file in your
home directory to include it in your local CPAN mirror.
also_mirror: indices/find-ls.gz
Note that CPAN::Mini does not mirror developer versions. Therefore, a live,
network CPAN Mirror will be needed in the urllist to retrieve these.
Note that CPAN requires the LWP module to be installed to use a local CPAN
mirror.
Alternatively, you might experiment with the alpha-quality release of
CPAN::Mini::Devel, which subclasses CPAN::Mini to retrieve developer
distributions (and find-ls.gz) using the same logic as CPAN::Reporter::Smoker.
Timing out hanging tests¶
CPAN::Reporter (since 1.08) supports a 'command_timeout' configuration option.
Set this option in the CPAN::Reporter configuration file to time out tests
that hang up or get stuck at a prompt. Set it to a high-value to avoid timing
out a lengthy tests that are still running -- 1000 or more seconds is probably
enough.
Warning -- on Win32, terminating processes via the command_timeout is equivalent
to SIGKILL and could cause system instability or later deadlocks
This option is still considered experimental.
Avoiding repetitive prerequisite testing¶
Because CPAN::Reporter::Smoker satisfies all requirements from scratch, common
dependencies (e.g. Class::Accessor) will be unpacked, built and tested
repeatedly.
As of version 1.92_56, CPAN supports the "trust_test_report_history"
config option. When set, CPAN will check the last test report for a
distribution. If one is found, the results of that test are used instead of
running tests again.
$ cpan
cpan> o conf init trust_test_report_history
cpan> o conf commit
Avoiding repetitive prerequisite builds (EXPERIMENTAL)¶
CPAN has a "build_dir_reuse" config option. When set (and if a YAML
module is installed and configured), CPAN will attempt to make build
directories persistent. This has the potential to save substantial time and
space during smoke testing. CPAN::Reporter::Smoker will recognize if this
option is set and make adjustments to the test process to keep PERL5LIB from
growing uncontrollably as the number of persistent directories increases.
NOTE: Support for "build_dir_reuse" is highly experimental.
Wait for at least CPAN version 1.92_62 before trying this option.
$ cpan
cpan> o conf init build_dir_reuse
cpan> o conf commit
Stopping early if a prerequisite fails¶
Normally, CPAN.pm continues testing a distribution even if a prerequisite fails
to build or fails testing. Some distributions may pass their tests even
without a listed prerequisite, but most just fail (and CPAN::Reporter discards
failures if prerequisites are not met).
As of version 1.92_57, CPAN supports the "halt_on_failure" config
option. When set, a prerequisite failure stops further processing.
$ cpan
cpan> o conf init halt_on_failure
cpan> o conf commit
However, a disadvantage of halting early is that no DISCARD grade is recorded in
the history. The next time CPAN::Reporter::Smoker runs, the distribution will
be tested again from scratch. It may be better to let all prerequisites finish
so the distribution can fail its test and be flagged with DISCARD so it will
be skipped in the future.
CPAN cache bloat¶
CPAN will use a lot of scratch space to download, build and test modules. Use
CPAN's built-in cache management configuration to let it purge the cache
periodically if you don't want to do this manually. When configured, the cache
will be purged on start and after a certain number of distributions have been
tested as determined by the "clean_cache_after" option for the
"start()" function.
$ cpan
cpan> o conf init build_cache scan_cache
cpan> o conf commit
CPAN verbosity¶
Recent versions of CPAN are verbose by default, but include some lesser known
configuration settings to minimize this for untarring distributions and for
loading support modules. Setting the verbosity for these to 'none' will
minimize some of the clutter to the screen as distributions are tested.
$ cpan
cpan> o conf init /verbosity/
cpan> o conf commit
Saving reports to files instead of sending directly¶
In some cases, such as when smoke testing using a development or prerelease
toolchain module like Test-Harness, it may be preferable to save reports to
files in a directory for review prior to submitting them. To do this, manually
set the "transport" option in your CPAN::Reporter config file to use
the Test::Reporter::Transport::File transport.
transport=File /path/to/directory
After review, send saved reports using Test::Reporter:
Test::Reporter->new()->read($filename)->send()
ENVIRONMENT¶
Automatically sets the following environment variables to true values while
running:
- •
- "AUTOMATED_TESTING" -- signal that tests are being run by an
automated smoke testing program (i.e. don't expect interactivity)
- •
- "PERL_MM_USE_DEFAULT" -- accept ExtUtils::MakeMaker
prompt() defaults
- •
- "PERL_EXTUTILS_AUTOINSTALL" -- set to '--defaultdeps' for
default dependencies
The following environment variables, if set, will modify the behavior of
CPAN::Reporter::Smoker. Generally, they are only required during the testing
of CPAN::Reporter::Smoker
- •
- "PERL_CR_SMOKER_RUNONCE" -- if true, "start()" will
exit after all distributions are tested instead of sleeping for the
"restart_delay" and then continuing
- •
- "PERL_CR_SMOKER_SHORTCUT" -- if true, "start()" will
process arguments (if any) but will return before starting smoke testing;
used for testing argument handling by "start()"
BUGS¶
Please report any bugs or feature using the CPAN Request Tracker. Bugs can be
submitted through the web interface at
<
http://rt.cpan.org/Dist/Display.html?Queue=CPAN-Reporter-Smoker>
When submitting a bug or request, please include a test-file or a patch to an
existing test-file that illustrates the bug or desired feature.
SEE ALSO¶
- •
- CPAN
- •
- CPAN::Reporter
- •
- CPAN::Testers
- •
- CPAN::Mini
- •
- CPAN::Mini::Devel
SUPPORT¶
Bugs / Feature Requests¶
Please report any bugs or feature requests through the issue tracker at
<
https://github.com/cpan-testers/CPAN-Reporter-Smoker/issues>. You will
be notified automatically of any progress on your issue.
Source Code¶
This is open source software. The code repository is available for public review
and contribution under the terms of the license.
<
https://github.com/cpan-testers/CPAN-Reporter-Smoker>
git clone https://github.com/cpan-testers/CPAN-Reporter-Smoker.git
AUTHOR¶
David Golden <dagolden@cpan.org>
CONTRIBUTORS¶
- •
- Alexandr Ciornii <alexchorny@gmail.com>
- •
- Christian Walde <walde.christian@googlemail.com>
COPYRIGHT AND LICENSE¶
This software is Copyright (c) 2013 by David Golden.
This is free software, licensed under:
The Apache License, Version 2.0, January 2004