table of contents
- NAME
- SYNOPSIS
- LONG OPTION EQUIVALENTS
- DESCRIPTION
- MODE 1 - stdout mode
- MODE 2 - Multifile mode
- MODE 3 - Batch mode
- SUCK ARGUMENT FILE
- SUCKNEWSRC
- SUCKKILLFILE and SUCKXOVER
- SUCKKILLFILE and GROUP KEEP/KILLFILES
- PARAMETERS
- KILL/KEEP Files Parameters
- SUCKXOVER
- SUCKXOVER and PROGRAM= or PERL= parameters
- SUCKOTHERMSGS
- SUCKNODOWNLOAD
- POST FILTER
- FOREIGN LANGUAGE PHRASES
- SIGNAL HANDLING
- EXIT CODES
- HISTORY
- SEE ALSO
SUCK(1) | General Commands Manual | SUCK(1) |
NAME¶
suck - Pull a small newsfeed from an NNTP server, avoiding the NEWNEWS command.SYNOPSIS¶
suck [ hostname ] [ @filename ] [ -V ] [ -K ] [ -L[SL] ] [ -LF filename ] [ -H ] [ -HF filename ] [ -d[tmd] dirname ] [ -s | -S filename ] [ -e | -E filename ] [ -a ] [ -m ] [ -b[irlf] batchfile ] [ -r filesize ] [ -p extension ] [ -U userid ] [ -P password ] [ -Q ] [ -c ] [ -M ] [ -N port_number ] [ -W pause_time pause_nr_msgs ] [ -w pause_time pause_nr_msgs ] [ -l phrase_file ] [ -D ] [ -R ] [ -q ] [ -C count ] [ -k ] [ -A ] [ -AL activefile ] [ -hl localhost ] [ -bp ] [ -T timeout ] [ -n ] [ -u ] [ -z ] [ -x ] [ -B ] [ -O ] [ -G ] [ -X ] [ -f ] [ -y post_filter ] [ -F ] [ -g ] [ -i number_to_read ] [ -Z ] [ -rc ] [ -lr ] [ -sg ] [ -ssl ] [ -SSL ]1. Moving sucknewsrc to sucknewsrc.old
2. Moving suck.newrc to sucknewsrc
3. rm suck.sorted and suckothermsgs.
LONG OPTION EQUIVALENTS¶
-a --always_batch
-bi --batch-inn
-br --batch_rnews
-bl --batch_lmove
-bf --batch_innfeed
-bp --batch_post
-c --cleanup
-dt --dir_temp
-dd --dir_data
-dm --dir_msgs
-e --def_error_log
-f --reconnect_dedupe
-g --header_only
-h --host
-hl --localhost
-k --kill_no_postfix
-l --language_file
-lr --low_read
-m --multifile
-n --number_mode
-p --postfix
-q --quiet
-r --rnews_size
-rc --resetcounter
-s --def_status_log
-sg --show_group
-ssl --use_ssl
-w --wait_signal
-x --no_chk_msgid
-y --post_filter
-z --no_dedupe
-A --active
-AL --read_active
-B --pre-batch
-C --reconnect
-D --debug
-E --error_log
-G --use_gui
-H --no_history
-HF --history_file
-K --killfile
-L --kill_log_none
-LS --kill_log_short
-LL --kill_log_long
-M --mode_reader
-N --portnr
-O --skip_on_restart
-P --password
-Q --password_env
-R --no_rescan
-S --status_log
-SSL --local_use_ssl
-T --timeout
-U --userid
-V --version
-W --wait
-X --no_xover
-Z --use_xover
DESCRIPTION¶
MODE 1 - stdout mode¶
%suck
%suck myhost.com
Suck grabs news from an NNTP server and sends the articles to stdout. Suck
accepts as argument the name of an NNTP server or if you don't give an
argument it will take the environment variable NNTPSERVER. You can redirect
the articles to a file or compress them on the fly like "suck
server.domain | gzip -9 > output.gz". Now it's up to you what you do
with the articles. Maybe you have the output already on your local machine
because you used a slip line or you still have to transfer the output to your
local machine.
MODE 2 - Multifile mode¶
%suck -m
%suck myhost.com -m
Suck grabs news from an NNTP server and stores each article in a separate file.
They are stored in the directory specified in suck_config.h or by the -dm
command line option.
MODE 3 - Batch mode¶
%suck myhost.com -b[irlf] batchfile
or %suck myhost.com -bp -hl localhost
or %suck myhost.com -bP NR -hl localhost
%suck myhost.com -b[irlf] batchfile
Suck will grab news articles from an NNTP server and store them into files, one
for each article (Multifile mode). The location of the files is based on the
defines in suck_config.h and the command line -dm. Once suck is done
downloading the articles, it will build a batch file which can be processed by
either innxmit or rnews, or it will call lmove to put the files directly into
the news/group/number format.
%innxmit localhost batchfile
The configuration file name (the batchfile
name provided with this option)
The directory specified for articles (-dm or
built-in default).
The errorlog to log errors to (-e or -E), if
provided on the command line.
The phrases file (-l), if provided on the
command line.
The Debug option, if provided on the command
line.
%rnews -S localhost batchfile
%suck news.server.com -bp -hl localhost -A
-c
%suck news.server.com -bP 100 -hl localhost -A
-c
SUCK ARGUMENT FILE¶
If you specify @filename on the command line, suck will read from filename and parse it for any arguments that you wish to pass to suck. You specify the same arguments in this file as you do on the command line. The arguments can be on one line, or spread out among more than one line. You may also use comments. Comments begin with '#' and go to the end of a line. All command line arguments override arguments in the file.# Sample Argument file
-bi batch # batch file option
-M # use mode reader option
SUCKNEWSRC¶
Suck looks for a file sucknewsrc to see what articles you want and which you already received. The format of sucknewsrc is very simple. It consists of one line for each newsgroup. The line contains two or three fields.comp.os.linux.announce 1 [ 100 ]
When suck is finished, it creates the file suck.newrc which contains the new
sucknewsrc with the updated article numbers.
To add a new newsgroup, just stick it in sucknewsrc, with a highest article
number of -1 (or any number less than 0). Suck will then get the newest X
number of messages for that newsgroup. For example, a -100 would cause suck to
download the newest 100 articles for that newsgroup.
To tell suck to skip a newsgroup, put a # as the first character of a line.
SUCKKILLFILE and SUCKXOVER¶
There are two types of killfiles supported in suck. The first, via the file suckkillfile, kills articles based on information in the actual article header or body. The second, via the file suckxover, kills articles based on the information retreived via the NNTP command XOVER. They are implemented in two fundamentally different ways. The suckkillfile killing is done as the articles are downloaded, one at a time. The XOVER killing is done while suck is getting the list of articles to download, and before a single article is downloaded. You may use either, none or both type of killfiles.SUCKKILLFILE and GROUP KEEP/KILLFILES¶
If suckkillfile exists, the headers of all articles will be scanned and the article downloaded or not, based on the parameters in the files. If no logging option is specified (see the -L options above), then the long logging option is used. Comments lines are allowed in the killfiles. A comment line has a "#" in the first position. Everything on a comment line is ignored. Here's how the whole keep/delete package works. All articles are checked against the master kill file (suckkillfile). If an article is not killed by the master kill file, then its group line is parsed. If a group file exists for one of the groups then the article is checked against that group file. If it matches a keep file, then it is kept, otherwise it is flagged for deletion. If it matches a delete file, then it is flagged for deletion, otherwise it is kept. This is done for every group on the group line. NOTES: With the exception of the USE_EXTENDED_REGEX parameter, none of these parameters are passed from the master killfile to the individual group file. Each killfile is separate and independant. Also, each search is case-insensitive unless specifically specified by starting the search string with the QUOTE character (see below). However, the parameter part of the search expression (the LOWLINE=, HILINE= part) is case sensitive.PARAMETERS¶
LOWLINES=#######
HILINES=#######
NRGRPS=####
NRXREF=####
QUOTE=c
NON_REGEX=c
GROUP=keep groupname filename OR GROUP=delete
groupname filename
PROGRAM=pathname
PERL=pathname
TIEBREAKER_DELETE
GROUP_OVERRIDE_MASTER
USE_EXTENDED_REGEX
XOVER_LOG_LONG
HEADER:
Any Valid Header Line:
BODY:
BODYSIZE>
BODYSIZE<
KILL/KEEP Files Parameters¶
HILINES= Match any article longer than the number of lines specified. LOWLINES= Match any article shorter than the number of lines specified. NRGRPS= This line will match any article which has more groups than the number specified on the Newsgroups: line. Typically this is used in a killfile to prevent spammed articles. (A spammed article is one that is posted to many many groups, such as those get-rich quick schemes, etc.) NRXREF= This line will match any article that has more groups than than the number specified on the Xref: line. This is another spamm stopper. WARNING: the Xref: line is not as accurate as the Newsgroups: line, as it only contains groups known to the news server. This option is most useful in an xover killfile, as in Xoverviews don't typically provide the Newsgroups: line, but do provide the Xref: line. HEADER: Any Valid Header Line: Suck allows you to scan any single header line for a particular pattern/string, or you may scan the entire article header. To scan an individual line, just specify it, for example to scan the From line for boby@pixi.com, you would putFrom:boby@pixi.com
From:boby@xxx
From:nerd@yyy
Subject:suck
Subject:help
The parameter HEADER: is a special case of the above. If you use the HEADER:
parameter, then the entire header is searched for the item. You are allowed
multiple HEADER: lines in each killfile.
When suck searches for the pattern, it only searches for what follows the :, and
spaces following the : are significant. With the above example
"Subject:suck", we will search the Subject header line for the
string "suck". If the example had read "Subject: suck",
suck would have searched for the string " suck". Note the extra
space.
If your system has regex() routines on it, then the items searched for can be
POSIX regular expressions, instead of just strings. Note that the QUOTE=
option is still applied, even to regular expressions.
BODY: This parameter allows you to search the body of an article for
text. Again, if your system has regex(), you can use regular expressions, and
the QUOTE= option is also applied. You are allowed multiple BODY: lines in
each killfile. WARNING: Certain regex combinations, especially with .* at the
beginning, (eg BODY:.*jpg), in combination with large articles, can cause the
regex code to eat massive amounts of CPU, and suck will seem like it is doing
nothing.
BODYSIZE> This parameter will match an article if the size of its body
(not including the header) is greater than this parameter. The size is
specified in bytes.
BODYSIZE< This parameter will match an article if the size of its
body, is less than this parameter. The size is specified in bytes.
QUOTE= This item specifies the character that defines a quoted string.
The default for this is a ". If an item starts with the QUOTE character,
then the item is checked as-is (case significant). If an item does not start
with the QUOTE character, then the item is checked with out regard to case.
NON_REGEX= This items specifies the character that defines a non-regex
string. The default for this is a %. If an item starts with the NON_REGEX
character, then the item is never checked for regular expressions. If the item
doesn't start with the QUOTE character, then suck tries to determine if it is
a regular expression, and if it is, use regex() on it. This item is so that
you can tell suck to treat strings like "$$$$ MONEY $$$$" as
non-regex items. IF YOU USE BOTH QUOTE and NON_REGEX characters on a string,
the NON_REGEX character MUST appear first.
GROUP= This line allows you to specify either keep or delete parameters
on a group by group basis. There are three parts to this line. Each part of
this line must be separated by exactly one space. The first part is either
"keep" or "delete". If it is keep, then only articles in
that group which match the parameters in the group file are downloaded. If it
is delete, articles in that group which match the parameters are not
downloaded. The second part, the group name is the full group name for
articles to check against the group file. The group name may contain an * as
the last character, to match multiple groups, eg: "comp.os.linux.*"
would match comp.os.linux.announce, comp.os.linux.answers, etc.. The third
part specifies the group file which contains the parameters to check the
articles against. Note, that if you specified a postfix with the -p option,
then this postfix is attached to the name of the file when suck looks for it,
UNLESS you use the -k option above.
GROUP_OVERRIDE_MASTER This allows you to override the default behavior of
the master kill file. If this option is in the master kill file, then even if
an article is flagged for deletion by the master kill file, it is checked
against the group files. If the group files says to not delete it, then the
article is kept.
TIEBREAKER_DELETE This option allows you to override the built-in
tie-breaker default. The potential exists for a message to be flagged by one
group file as kept, and another group file as killed. The built-in default is
to then keep the message. The TIEBREAKER_DELETE option will override that, and
caused the article to be deleted.
USE_EXTENDED_REGEX This option tells suck to use extended regular
expressions vice standard regular expressions. It may used in the master
killfile, in which case it applies to all killfiles, or in an individual
killfile, where it only applies to the parameters that follow it in the
killfile.
XOVER_LOG_LONG This option tells suck to format the killfile generated by
from an Xover killfile so that it looks like an article header. The normal
output is to just print the Xover line from theserver.
PROGRAM= This line allows suck to call an external program to check each
article. You may specify any arguments in addition to the program name on this
line. If this line is in your suckkillfile, all other lines are ignored.
Instead, the headers are passed to the external program, and the external
program determines whether or not to download the article. Here's how it
works. Suck will fork your program, with stdin and stdout redirected. Suck
will feed the headers to your program thru stdin, and expect a reply back thru
stdout. Here's the data flow for each article:
1. suck will write a 8 byte long string, which
represents the length of the header record on stdin of the external program.
Then length is in ascii, is left-aligned, and ends in a newline (example:
"1234 \n").
2. suck will then write the header on stdin of
the external program.
3. suck will wait for a 2 character response
code on stdout. This response code is either "0\n" or
"1\n" (NOT BINARY ZERO OR ONE, ASCII ZERO OR ONE). If the return
code is zero, suck will download the article, if it is one, suck won't.
4. When there are no more articles, the length
written down (for step 1) will be zero (again in ascii "0 \n"). Suck
will then wait for the external program to exit before continuing on. The
external program can do any clean up it needs, then exit. Note: suck will not
continue processing until the external program exits.
PERL=perl_kill.pl
SUCKXOVER¶
If the file suckxover exists, then suck uses the XOVER command to get information on the articles and decide whether or not to download the article. Xover files use the same syntax as suckkillfiles, but supports a subset of the commands. The following killfile commands are not supported in suckxover files:NRGROUPS:
HEADER:
BODY:
TIEBREAKER_DELETE:
Only the following header lines will be checked:
Subject:
From:
Message-ID:
References:
The behaviour of the size commands ( BODYSIZE>, BODYSIZE<, HILINES, and
LOWLINES ) specify the total size of the article (not just the body) in
bytes or lines, respectively.
All other parameters are allowed. However, if you use an invalid parameter, it
is silently ignored.
SUCKXOVER and PROGRAM= or PERL= parameters¶
These parameters are supported in a suckxover file, however they work slightly differently than described above. The key difference is that prior to sending each individual xoverview line to your program, suck will send you the overview.fmt listing that it retrieves from the server. This overview.fmt is a tab-separated line, describing the fields in each overview.fmt line. For the PROGRAM= parameter, suck will first send your program an 8 byte long string, which is the length of the overview.fmt. This length is formatted as the lengths above (see nr1 under PROGRAM=). Suck will then send the overview.fmt. After that, the flow is as described above. See sample/killxover_child.c for an example. For the PERL= parameter, Your program must have two subroutines. The first is perl_overview, which will recieve the overview.fmt, and not return anything. The second subroutine is perl_xover, which will recieve the xoverview line, and return 0 or 1, as described in the PERL= above. See sample/perl_xover.pl for an example.SUCKOTHERMSGS¶
If suckothermsgs exists, it must contain lines formatted in one of three ways. The first way is a line containing a Message-ID, with the <> included, eg: <12345@somehost.com>
!comp.os.linux.announce 1
!comp.os.linux.announce 1-10
Whichever method you use, if the article specified exists, it will be
downloaded, in addition to any articles retreived via the sucknewsrc.
These ways can be used to get a specific article in other groups, or to
download an article that was killed. These articles ARE NOT processed
through the kill articles routines.
SUCKNODOWNLOAD¶
If sucknodownload exists, it must consist of lines contaning a Message-ID, with the <> included, eg: <12345@somehost.com>
POST FILTER¶
if the -y post_filter option is specified on the command line in conjunction with any of the batch modes, then suck will call the post filter specified, after downloading the articles, and before batching/posting the articles. The filter is passed the directory where the articles are stored (the -dm option). The filter program is responsible for parsing the contents of the directory. See sample/post_filter.pl for a sample post filter. This option was designed to allow you to add your own host name to the Path: header, but if you need to do anything else to the messages, you can.FOREIGN LANGUAGE PHRASES¶
If the -l phrases option is specified or the file /usr/local/lib/suck.phrases (defined in suck_config.h) exists, then suck will load an alternate language phrase file, and use it for all status & error messages, instead of the built-in defaults. The command line overrides the build in default, if both are present. The phrase file contains all messages used by suck, rpost, testhost, and lmove, each on a separate line and enclosed in quotes. To generate a sample phrase file, run make phrases from the command line. This will create "phrases.engl", which is a list of the default phrases. Simply edit this file, changing the english phrases to the language of your choosing, being sure to keep the phrases within the quotes. These phrases may contain variables to print items provided by the program, such as hostname. Variables are designated by %vN% where N is a one-up sequence per phrase. These variables may exist in any order on the phrase line, for example,"Hello, %v1%, welcome to %v2%"
or
"Welcome to %v2%, %v1%"
are both valid phrases. Phrases may contain, \n, \r, or \t to print a newline,
carriage return, or tab, respectively. Note that the first line of the phrase
file is the current version number. This is checked against the version of
suck running, to be sure that the phrases file is the correct version.
SIGNAL HANDLING¶
Suck accepts two signals, defined in suck_config.h. The first signal (default SIGTERM) will cause Suck to finish downloading the current article, batch up whatever articles were downloaded, and exit, without an error.EXIT CODES¶
Suck will exit with the following return codes:0 = success
1 = no articles available for download.
2 = suck got an unexpected answer to a command
it issued to the remote server.
3 = the -V option was used.
4 = suck was unable to perform NNTP
authorization with the remote server.
-1 = general error.
HISTORY¶
Original Author - Tim Smith (unknown
address)
Maintainers -
March 1995 - Sven Goldt
(goldt@math.tu-berlin.de)
July 1995 - Robert A. Yetman
(boby@pixi.com)