.\" Automatically generated by Pod::Man 4.07 (Pod::Simple 3.32) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .if !\nF .nr F 0 .if \nF>0 \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} .\} .\" ======================================================================== .\" .IX Title "FileList 3pm" .TH FileList 3pm "2015-12-10" "perl v5.24.1" "User Contributed Perl Documentation" .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" File::RsyncP::FileList \- Perl interface to rsync file list encoding and decoding. .SH "SYNOPSIS" .IX Header "SYNOPSIS" .Vb 1 \& use File::RsyncP::FileList; \& \& $fileList = File::RsyncP::FileList\->new({ \& preserve_uid => 1, \& preserve_gid => 1, \& preserve_links => 1, \& preserve_devices => 1, \& preserve_hard_links => 0, \& always_checksum => 0, \& remote_version => 26, \& }); \& \& # decoding an incoming file list \& while ( !$fileList\->decodeDone && !$fileList\->fatalError ) { \& $data .= readMoreDataFromRemoteRsync(); \& $bytesDone = $fileList\->decode($data); \& $data = substr($data, $bytesDone) if ( $bytesDone > 0 ); \& } \& $fileList\->clean; \& \& # create (encode) a file list \& $fileList\->encode({ \& name => $filePath, \& dev => $stat[0], \& inode => $stat[1], \& mode => $stat[2], \& uid => $stat[4], \& gid => $stat[5], \& rdev => $stat[6], \& mtime => $stat[9], \& }); \& $data = $fileList\->encodeData; \& \& # get file information, for file number 5: \& $fileInfo = $fileList\->get(5); \& \& # utility functions \& $numberOfFiles = $fileList\->count; \& $gotFatalError = $fileList\->fatalError; .Ve .SH "DESCRIPTION" .IX Header "DESCRIPTION" The File::RsyncP::FileList module is used to encode and decode file lists in using the same format at Rsync. .PP The sender side of Rsync creates a list of all the files the are going to be sent. This list is sent in a compact format to the receiver side. Each side then sorts the list and removes duplicate entries. From this point on, all files are referred to by their integer index into the sorted file list. .PP A new file list object is created by calling File::RsyncP::FileList\->new. An object can be used to decode or encode a file list. There is no mechanism to reset the state of a file list: you should create a new object each time you need to do a new decode or encode. .PP The \fInew()\fR function takes a hashref of options, which correspond to various rsync command-line switches. These must exactly match the arguments to the remote rsync, otherwise the file list format will not be compatible and decoding will fail. .PP .Vb 9 \& $fileList = File::RsyncP::FileList\->new({ \& preserve_uid => 1, # \-\-owner \& preserve_gid => 1, # \-\-group \& preserve_links => 1, # \-\-links \& preserve_devices => 1, # \-\-devices \& preserve_hard_links => 0, # \-\-hard\-links \& always_checksum => 0, # \-\-checksum \& remote_version => 26, # remote protocol version \& }); .Ve .SS "Decoding" .IX Subsection "Decoding" The decoding functions take a stream of bytes from the remote rsync and convert them into an internal data structure. Rather than store the file list as a native perl list of hashes (which occupies too much memory for large file lists), the same internal data structure as rsync is used. Individual file list entries can be returned with the \fIget()\fR function. .PP File list data read from the remote rsync should be passed to the \&\fIdecode()\fR function. The data may be read and processed in arbitrary sized chunks. The \fIdecode()\fR function returns how many bytes were actually processed. It is the caller's responsbility to remove that number of bytes from the input argument, preserving the remaining bytes for the next call to \fIdecode()\fR. The \fIdecodeDone()\fR function returns true when the file list is complete. The \fIfatalError()\fR function returns true if there was a non-recoverable error while decoding. .PP The \fIclean()\fR function needs to be called after the file list decode is complete. The \fIclean()\fR function sorts the file list and removes repeated entries. Skipping this step will produce unexpected results: since files are referred to using integers, each side will refer to different files is the file lists are not sorted and purged in exactly the same manner. .PP A typical decode loop looks like: .PP .Vb 6 \& while ( !$fileList\->decodeDone && !$fileList\->fatalError ) { \& $data .= readMoreDataFromRemoteRsync(); \& $bytesDone = $fileList\->decode($data); \& $data = substr($data, $bytesDone) if ( $bytesDone > 0 ); \& } \& $fileList\->clean; .Ve .PP After \fIclean()\fR is called, the number of files in the file list can be found by calling \fIcount()\fR. Files can be fetched by calling the \fIget()\fR function, with an index from 0 to \fIcount()\fR\-1: .PP .Vb 1 \& $fileInfo = $fileList\->get(5); .Ve .PP The \fIget()\fR function returns a hashref with various entries: .PP .Vb 10 \& name path name of the file (relative to rsync dir): \& equal to dirname/basename \& basename file name, without directory \& dirname directory where file resides \& sum file MD4 checksum (only present if \-\-checksum specified) \& uid file user id \& gid file group id \& mode file mode \& mtime file modification time \& size file length \& dev device number on which file resides \& inode file inode \& link link contents if the file is a sym link \& rdev major/minor device number if file is char/block special .Ve .PP Various fields will only have valid values if the corresponding options are set (eg: uid if preserve_uid is set, dev and inode if preserve_hard_links is set etc). .PP For example, to dump out each of hash you could do this: .PP .Vb 6 \& use Data::Dumper; \& my $count = $fileList\->count; \& for ( my $i = 0 ; $i < $count ; $i++ ) { \& print("File $i is:\en"); \& print Dumper($fileList\->get($i)); \& } .Ve .SS "Encoding" .IX Subsection "Encoding" The \fIencode()\fR function is used to build a file list in preparation for encoding and sending a file list to a remote rsync. The \fIencode()\fR function takes a hashref argument with the parameters for one file. It should be called once for each file. The parameter names are the same as those returned by \fIget()\fR. .PP In this example the matching \fIstat()\fR values are shown: .PP .Vb 11 \& $fileList\->encode({ \& name => $filePath, \& dev => $stat[0], \& inode => $stat[1], \& mode => $stat[2], \& uid => $stat[4], \& gid => $stat[5], \& rdev => $stat[6], \& size => $stat[7], \& mtime => $stat[9], \& }); .Ve .PP It is not necessary to specify basename and dirname; these are extracted from name. You only need to specify the parameters that match the options given to \fInew()\fR. You can also specify sum and link as necessary. .PP To compute the encoded file list data the \fIencodeData()\fR function should be called. It can be called every time \fIencode()\fR is called, or once at the end of all the \fIencode()\fR calls. It returns the encoded data that should be sent to the remote rsync: .PP .Vb 1 \& $data = $fileList\->encodeData; .Ve .PP It is recommended that \fIencodeData()\fR be called frequently to avoid the need to allocate large internal buffers to hold the entire encoded file list. Since \fIencodeData()\fR does not know when the last file has been encoded, it is the caller's responsbility to add the final null byte (eg: pack(\*(L"C\*(R", 0)) to the data to indicate the end of the file list data. .PP After all the file list entries are processed you should call \fIclean()\fR: .PP .Vb 1 \& $fileList\->clean; .Ve .PP This ensures that each side (sender/receiver) has identical sorted file lists. .SS "Utility functions" .IX Subsection "Utility functions" The \fIcount()\fR function returns the total number of files in the internal file list (either decoded or encoded). .PP The \fIfatalError()\fR function returns true if a fatal error has occured during file decoding. It should be called in the decode loop to make sure no error has occured. .SH "AUTHOR" .IX Header "AUTHOR" File::RsyncP::FileList was written by Craig Barratt based on rsync 2.5.5. .PP Rsync was written by Andrew Tridgell and Paul Mackerras. It is available under a \s-1GPL\s0 license. See http://rsync.samba.org .SH "LICENSE" .IX Header "LICENSE" This program is free software; you can redistribute it and/or modify it under the terms of the \s-1GNU\s0 General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. .PP This program is distributed in the hope that it will be useful, but \s-1WITHOUT ANY WARRANTY\s0; without even the implied warranty of \&\s-1MERCHANTABILITY\s0 or \s-1FITNESS FOR A PARTICULAR PURPOSE.\s0 See the \s-1GNU\s0 General Public License for more details. .PP You should have received a copy of the \s-1GNU\s0 General Public License in the \&\s-1LICENSE\s0 file along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, \s-1MA 02111\-1307 USA.\s0 .SH "SEE ALSO" .IX Header "SEE ALSO" See for File::RsyncP's SourceForge home page. .PP See File::RsyncP and File::RsyncP::FileIO for more precise examples of using File::RsyncP::FileList. .PP Also see BackupPC's lib/BackupPC/Xfer/RsyncFileIO.pm for other examples.