.\" -*- mode: troff; coding: utf-8 -*- .\" Automatically generated by Pod::Man 5.01 (Pod::Simple 3.43) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" \*(C` and \*(C' are quotes in nroff, nothing in troff, for use with C<>. .ie n \{\ . ds C` "" . ds C' "" 'br\} .el\{\ . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" ======================================================================== .\" .IX Title "BUFFINDEXED.CONF 5" .TH BUFFINDEXED.CONF 5 2024-02-12 "INN 2.7.2" "InterNetNews Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH NAME buffindexed.conf \- Configuration for the buffindexed overview method .SH DESCRIPTION .IX Header "DESCRIPTION" \&\fIbuffindexed.conf\fR, found in \fIpathetc\fR, specifies the buffers that the buffindexed overview method should use. It is required if the server uses buffindexed (as configured by the \fIovmethod\fR parameter in \fIinn.conf\fR). .PP Buffindexed uses pre-built buffer files to store overview data and indexes to that data. The buffers are divided into 8\ KB internally, and a given block is used either for overview data or for index data. A block is always allocated to a single newsgroup and is never shared among newsgroups. It also means that overview data is limited to 8\ KB per article, which may lead to the lack of integration of a few articles with headers of unusual length into the overview database. .PP In addition to the buffers, buffindexed also stores information in a file named \fIgroup.index\fR in \fIpathdb\fR. (This file should not be mistaken for the one named \fIgroup.index\fR in \fIpathoverview\fR which is used by the tradindexed overview method.) It contains information about each newsgroup: the pointer to the index block for the newsgroup, the high mark, the low mark, the flag of the group, the number of articles, and so forth. This file is created automatically when all buffers are initialized and should not be manually edited. .PP Buffindexed buffers are of fixed size, so buffindexed will never use more space than what is available in those buffers. If all buffers are full, \&\fBinnd\fR will throttle when it attempts to store overview information for any additional articles until space is freed (with \fBexpireover\fR, for instance) or another buffer is added. This is unlike the CNFS storage method. .PP You can see the current usage of the buffers with the \fB\-o\fR option to \&\fBinndf\fR. .PP The overview data of received articles are flushed to disk every \&\fIovflushcount\fR articles, as set in \fIinn.conf\fR. .PP In the \fIbuffindexed.conf\fR file, blank lines and lines beginning with a number sign (\f(CW\*(C`#\*(C'\fR) are ignored. All other lines must be of the format: .PP .Vb 1 \& :: .Ve .PP The order of lines is not significant. .PP is the index of this overview buffer and must be unique. Other than that constraint, it can be any number between 0 and 65535. .PP is the path to the buffer. The length of the path should not be longer than 63 characters. .PP is the length of the buffer in kilobytes (1\ KB\ =\ 1024\ bytes). If does not specify a special device, the file size of the buffer must be \ *\ 1024\ bytes. If it does specify a special device, that device must have at least space available. For more information on setting up the buffers, see "CREATING BUFFERS". .PP An example of \fIbuffindexed.conf\fR file can be: .PP .Vb 2 \& 0:/OV1:1536000 \& 1:/OV2:1536000 .Ve .PP When you first start \fBinnd\fR with everything configured properly, you should see messages like this in \fIpathlog\fR/news.notice: .PP .Vb 2 \& Aug 27 00:00:00 kevlar innd: buffindexed: no magic cookie found \& for ovbuff 0, initializing .Ve .PP You MUST recreate overview completely using \fBmakehistory\fR if you remove or replace buffers. However, new buffers can be added without any special care (other than restarting \fBinnd\fR after modifying \fIbuffindexed.conf\fR). If you need to rebuild overview, you should zero all of the buffers first. .PP We recommend not to reserve too much spare space in existing buffers, so that to minimize the duration of the \fBexpireover\fR process, and to just add new buffers when space left is low (see the result of \f(CW\*(C`inndf \-no\*(C'\fR). Plan on needing at least 0.65\ KB for every article in your spool (not counting crossposts). So, if you have 5 million articles, you'll need at least 3.25\ GB of disk space for buffindexed. .SH "CREATING BUFFERS" .IX Header "CREATING BUFFERS" There are two methods to create a new buffindexed buffer: .IP 1. 4 Create a large file on top of a regular file system. The easiest way to do this is probably with dd(1), using a command like: .Sp .Vb 1 \& dd if=/dev/zero of=/path/to/cycbuff bs=1024 count= .Ve .Sp where is the size from the relevant line in \fIbuffindexed.conf\fR. .Sp This is the simplest method, but has the disadvantage that very large files on regular file systems can be fairly slow to access, particularly at the end of the file, and INN incurs unnecessary file system overhead when accessing the buffer. .IP 2. 4 Use block devices directly. If your operating system allows you to call mmap() on block devices (Solaris and recent versions of Linux do, FreeBSD at last report does not), this method can avoid all of the native file system overhead. Note, however, that Solaris has problems with byte range locking on block devices, and therefore this method should not be used on Solaris. .Sp Partition the disk. If you're using Solaris, set up your partitions to avoid the first cylinder of the disk (or otherwise the buffindexed header will overwrite the disk partition table and render the buffers inaccessible). Then, create device files for each block device you're going to use. .Sp It's not recommended to use the block device files in \fI/dev\fR, since the news system doesn't have permission to write to them and changing the permissions of the system device files may affect something else. Instead, use mknod(1) to create a new set of block devices (in somewhere like \fIpathspool\fR/overview that's only writable by the news user). To do this, run \f(CW\*(C`ls \-Ll\*(C'\fR on the devices in \fI/dev\fR that correspond to the block devices that you want to use. The major and minor device numbers are in the fifth and sixth columns (right before the date), respectively. Then run mknod like: .Sp .Vb 1 \& mknod b .Ve .Sp where is the path to the device to create (matching the part of the buffindexed configuration line) and and are the major and minor device numbers as discovered above. .Sp Here's a short script to do this when given the path to the system device file as an argument: .Sp .Vb 8 \& #!/bin/sh \& base=\`echo "$1" | sed \*(Aqs%.*/%%\*(Aq\` \& major=\`ls \-Ll "$1" | awk \*(Aq{print $5}\*(Aq | tr \-d ,\` \& minor=\`ls \-Ll "$1" | awk \*(Aq{print $6}\` \& mkdir \-p \& mknod /"$base" b "$major" "$minor" \& chown news:news /"$base" \& chmod 644 /"$base" .Ve .Sp Make sure that the created files are owned by the news user and news group, as specified at configure time (the default being \f(CW\*(C`news\*(C'\fR for both). Also make sure that the permissions on the devices allow the news user to read and write. .SH HISTORY .IX Header "HISTORY" Written by Katsuhiro Kondou for InterNetNews. Converted to POD by Russ Allbery . .SH "SEE ALSO" .IX Header "SEE ALSO" expireover(8), inn.conf(5), inndf(8), makehistory(8).