NAME¶
libstorage - routines for managing INN storage
SYNOPSIS¶
#include <inn/storage.h>
typedef enum {
RETR_ALL,
RETR_HEAD,
RETR_BODY,
RETR_STAT
} RETRTYPE;
typedef enum {
SM_RDWR,
SM_PREOPEN
} SMSETUP;
typedef unsigned char STORAGECLASS;
typedef unsigned char STORAGETYPE;
typedef struct token {
STORAGETYPE type;
STORAGECLASS class;
char token[STORAGE_TOKEN_LENGTH];
} TOKEN;
typedef struct {
unsigned char type;
const char *data;
struct iovec *iov;
int iovcnt;
size_t len;
unsigned char nextmethod;
void *private;
time_t arrived;
time_t expires;
char *groups;
int groupslen;
TOKEN *token;
} ARTHANDLE;
typedef enum {
SELFEXPIRE,
SMARTNGNUM,
EXPENSIVESTAT
} PROBETYPE;
typedef enum {
SM_ALL,
SM_HEAD,
SM_CANCELLEDART
} FLUSHTYPE;
struct artngnum {
char *groupname;
ARTNUM artnum;
};
bool IsToken(const char *text);
char *TokenToText(const TOKEN token);
TOKEN TextToToken(const char *text);
bool SMsetup(SMSETUP type, void *value);
bool SMinit(void);
TOKEN SMstore(const ARTHANDLE article);
ARTHANDLE *SMretrieve(const TOKEN token, const RETRTYPE amount);
ARTHANDLE *SMnext(const ARTHANDLE *article, const RETRTYPE amount);
void SMfreearticle(ARTHANDLE *article);
bool SMcancel(TOKEN token);
bool SMprobe(PROBETYPE type, TOKEN *token, void *value);
void SMprintfiles(FILE *file, TOKEN token, char **xref, int ngroups);
bool SMflushcacheddata(FLUSHTYPE type);
char *SMexplaintoken(const TOKEN token);
void SMshutdown(void);
int SMerrno;
char *SMerrorstr;
#include <inn/ov.h>
#define OV_NOSPACE ...
typedef enum {
OVSPACE,
OVSORT,
OVCUTOFFLOW,
OVGROUPBASEDEXPIRE,
OVSTATICSEARCH,
OVSTATALL,
OVCACHEKEEP,
OVCACHEFREE
} OVCTLTYPE;
typedef enum {
OVNEWSGROUP,
OVARRIVED,
OVNOSORT
} OVSORTTYPE;
typedef enum {
OVADDCOMPLETED,
OVADDFAILED,
OVADDGROUPNOMATCH
} OVADDRESULT;
bool OVopen(int mode);
bool OVctl(OVCTLTYPE type, void *val);
bool OVgroupstats(char *group, int *lo, int *hi, int *count, int *flag);
bool OVgroupadd(char *group, ARTNUM lo, ARTNUM hi, char *flag);
bool OVgroupdel(char *group);
OVADDRESULT OVadd(TOKEN token, char *data, int len, time_t arrived, time_t expires);
bool OVcancel(TOKEN token);
void *OVopensearch(char *group, int low, int high);
bool OVsearch(void *handle, ARTNUM *artnum, char **data, int *len, TOKEN *token, time_t *arrived);
void OVclosesearch(void *handle);
bool OVgetartinfo(char *group, ARTNUM artnum, TOKEN *token);
bool OVexpiregroup(char *group, int *lo, struct history *h);
typedef struct _OVGE {
bool delayrm;
bool usepost;
bool quiet;
bool keep;
bool earliest;
bool ignoreselfexpire;
char *filename;
time_t now;
float timewarp;
} OVGE;
void OVclose(void);
DESCRIPTION¶
libstorage is a library of common utility (the storage manager) routines
for accessing Usenet articles and related data independent of particular
storage method; this is known as the storage API.
The storage manager's function is to isolate the applications from the
individual methods and make the policy decisions as to which storage method
should be called to store an article. One of the core concepts in the storage
API is that articles are not manipulated using the message-ID or article
number, but rather a token that uniquely identifies the article to the method
that stored it. This may seem to be redundant since the message-ID already is
a unique identifier for the article; however, since the storage method
generates the token, it can encode all the information it needs to locate the
article in the token.
OV is a common utility routines for accessing newsgroups and overview
data independent of particular overview method.
The
OV function is to isolate the applications from the individual
methods. All articles passed through the storage API are assumed to be in wire
format. Wire format means "\r\n" at the end of lines, dot-stuffed
lines (that is to say "." at the beginning of lines which already
start with "."), and ".\r\n" at the end of article on NNTP
stream are not stripped. This has a performance win when transferring
articles. Note that for the tradspool method, wire format can be disabled.
This is just for compatibility which is needed by some old tools written for
traditional spool.
The
IsToken function checks to see if the text is formatted as a text
token string. It returns true if formatted correctly or returns false if not.
The
TokenToText function converts a token into a text string for output.
The
TextToToken function converts a text string into a token.
The
SMsetup function configures some parameters for use by the storage
manager.
type is one of following:
- "SM_RDWR"
- Allow read/write open for storage files (default is
false).
- "SM_PREOPEN"
- Open all storage files at startup time and keep them
(default is false).
value is the pointer which tells each type's value. It returns true if
setup is successful, or false if not.
The
SMinit function calls the setup function for all of the configured
methods based on
SMsetup. This function should be called prior to all
other storage API functions which begin with "SM" except
SMsetup. It returns true if initialization is successful or returns
false if not.
SMinit returns true, unless all storage methods fail
initialization.
The
SMstore function stores an article specified with
article. The
headers and body of the article are supplied to
SMstore using the
iov and
iovcnt members of
ARTHANDLE. (
data and
private are ignored by
SMstore.) If
arrived is specified,
SMstore uses its value as article's arrival time; otherwise
SMstore uses the current time for it.
SMstore returns the token
type or returns
TOKEN_EMPTY if the article is not stored because some
error occurs or simply does not match any
uwildmat(3) expression in
storage.conf.
SMstore fails if
SM_RDWR has not been set
to true with
SMsetup.
The
SMretrieve function retrieves an article specified with
token.
amount is the one of following which specifies retrieving type:
- "RETR_ALL"
- Retrieve the whole article.
- "RETR_HEAD"
- Retrieve the headers of the article.
- "RETR_BODY"
- Retrieve the body of the article.
- "RETR_STAT"
- Just check to see if the article exists.
SMretrieve provides the article data via the
data and
len
members of
ARTHANDLE. (
iov is not set by
SMretrieve.) The
data area indicated by
ARTHANDLE should not be modified.
The
SMnext function is similar in function to
SMretrieve except
that it is intended for traversing the method's article store sequentially. To
start a query,
SMnext should be called with a NULL pointer
ARTHANDLE. Then
SMnext returns
ARTHANDLE which should be
used for the next query. If a NULL pointer
ARTHANDLE is returned, no
articles are left to be queried. If
data of
ARTHANDLE is NULL
pointer or
len of
ARTHANDLE is 0, it indicates the article may
be corrupted and should be cancelled by
SMcancel. The data area
indicated by
ARTHANDLE should not be modified.
The
SMfreearticle function frees all allocated memory used by
SMretrieve and
SMnext. If
SMnext will be called with
previously returned
ARTHANDLE,
SMfreearticle should not be
called as
SMnext frees allocated memory internally.
The
SMcancel function removes the article specified with
token. It
returns true if cancellation is successful or returns false if not.
SMcancel fails if
SM_RDWR has not been set to true with
SMsetup.
The
SMprobe function checks the token on
PROBETYPE.
type is
one of following:
- "SELFEXPIRE"
- Check to see if the method of the token has self expire
functionality.
- "SMARTNGNUM"
- Get the newsgroup name and the article number of the
token.
- "EXPENSIVESTAT"
- Check to see whether checking the existence of an article
is expensive or not.
The
SMprintfiles function shows file name or token usable by
fastrm(8).
The
SMflushcacheddata function flushes cached data on each storage
method.
type is one of following:
- "SM_HEAD"
- Flush cached header.
- "SM_CANCELLEDART"
- Flush the articles which should be cancelled.
- "SM_ALL"
- Flush all cached data.
The
SMexplaintoken function returns a human-readable text string with a
clear, decoded form of the storage API token.
The
SMshutdown function calls the shutdown for each configured storage
method and then frees any resources it has allocated for itself.
SMerrno and
SMerrorstr indicate the reason of the last error
concerning storage manager.
OVopen calls the setup function for configured method which is specified
as
ovmethod in
inn.conf.
mode is constructed from
following:
- "OV_READ"
- Allow read open for the overview method.
- "OV_WRITE"
- Allow write open for the overview method.
This function should be called prior to all other OV functions which begin with
"OV".
The
OVctl function probes or sets some parameters for the overview
method.
type is one of following:
- "OVGROUPBASEDEXPIRE"
- Setup how group-based expiry is done.
- "OVCUTOFFLOW"
- Do not add overview data if the data is under the lowest
article.
- "OVSORT"
- Probe which key is suitable for the overview method.
- "OVSPACE"
- Probe the overview space usage.
- "OVSTATALL"
- Stat all the articles when OVexpiregroup is
called.
- "OVSTATICSEARCH"
- Setup if results of OVsearch are stored in a static
buffer and must be copied before the next call to OVsearch.
- "OVCACHEKEEP"
- Setup whether the cache should be kept.
- "OVCACHEFREE"
- Free the cache.
The
OVgroupstats function retrieves the specified newsgroup information
from the overview method.
The
OVgroupadd function informs the overview method that the specified
newsgroup is being added.
The
OVgroupdel function informs the overview method that the specified
newsgroup is being removed.
The
OVadd function stores an overview data.
The
OVcancel function requests the overview method delete overview data
specified with token.
The
OVopensearch function requests the overview method prepare overview
data retrieval. The request range is determined by
low and
high.
The setting of
OVSTATICSEARCH determines how search result data must be
handled. (Note that with some storage methods, each call to
OVopensearch may cause internal storage to be remapped. Therefore,
multiple simultaneous searches may require data to be copied in between
OVsearch calls even if
OVSTATICSEARCH is false.)
The
OVsearch function retrieves information, article number, overview
data, or arrival time. It should be called with NULL handle when it is the
first time; subsequent
OVsearch calls should use the handle returned by
the previous call to
OVsearch.
OVsearch returns true, unless it
reaches
high, which is specified by
OVopensearch. Retrieved
overview data are sorted by article number, and
len is 0 if there is no
overview data for the article. Note that the retrieved data is not necessarily
null-terminated; you should only rely on
len octets of overview data
being present.
The
OVclosesearch function frees all resources which have been allocated
by
OVopensearch.
The
OVgetartinfo function retrieves the overview data and the token
specified with
artnum.
The
OVexpiregroup function expires the overview data for the newsgroup.
It checks the existence of the article and purges the overview data if the
article no longer exists. If
groupbaseexpiry in
inn.conf is
true,
OVexpiregroup also expires articles.
The
OVclose function frees all resources which are used by the overview
method.
HISTORY¶
Written by Katsuhiro Kondou <kondou@nec.co.jp> for InterNetNews. Converted
to POD by Julien Elie.
$Id: libstorage.pod 9074 2010-05-31 19:01:32Z iulius $
SEE ALSO¶
expire(8),
fastrm(8),
inn.conf(5),
storage.conf(5).