table of contents
rsnapshot(1) | rsnapshot(1) |
NAME¶
rsnapshot - remote filesystem snapshot utilitySYNOPSIS¶
rsnapshot [-vtxqVD] [-c cfgfile] [command] [args]DESCRIPTION¶
rsnapshot is a filesystem snapshot utility. It can take incremental snapshots of local and remote filesystems for any number of machines. Local filesystem snapshots are handled with rsync(1). Secure remote connections are handled with rsync over ssh(1), while anonymous rsync connections simply use an rsync server. Both remote and local transfers depend on rsync. rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the size of one full backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use of hard links, so if the file doesn't change, the next snapshot is simply a hard link to the exact same file. rsnapshot will typically be invoked as root by a cron job, or series of cron jobs. It is possible, however, to run as any arbitrary user with an alternate configuration file. All important options are specified in a configuration file, which is located by default at /etc/rsnapshot.conf. An alternate file can be specified on the command line. There are also additional options which can be passed on the command line. The command line options are as follows:-v verbose, show shell commands being
executed
-t test, show shell commands that would be executed
-c path to alternate config file
-x one filesystem, don't cross partitions within each backup point
-q quiet, suppress non-fatal warnings
-V same as -v, but with more detail
-D a firehose of diagnostic information
CONFIGURATION¶
/etc/rsnapshot.conf is the default configuration file. All parameters in this file must be separated by tabs. /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz can be used as a reference. It is recommended that you copy /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz to /etc/rsnapshot.conf, and then modify /etc/rsnapshot.conf to suit your needs. Long lines may be split over several lines. "Continuation" lines must begin with a space or a tab character. Continuation lines will have all leading and trailing whitespace stripped off, and then be appended with an intervening tab character to the previous line when the configuation file is parsed. Here is a list of allowed parameters:config_version Config file version
(required). Default is 1.2
snapshot_root Local filesystem path to save all snapshots
include_conf Include another file in the configuration at this point.
This is recursive, but you may need to be
careful about paths when specifying which file to include. We check to see if
the file you have specified is readable, and will yell an error if it isn't.
We recommend using a full path. As a special case, include_conf's value may be
enclosed in `backticks` in which case it will be executed and whatever it
spits to STDOUT will be included in the configuration. Note that shell
meta-characters may be interpreted.
If you are using Linux, you should uncomment
cmd_cp. If you are using a platform which does not have GNU cp, you should
leave cmd_cp commented out.
With GNU cp, rsnapshot can take care of both normal files and special files
(such as FIFOs, sockets, and block/character devices) in one pass.
If cmd_cp is disabled, rsnapshot will use its own built-in function,
native_cp_al() to backup up regular files and directories. This will
then be followed up by a separate call to rsync, to move the special files
over (assuming there are any).
Full path (plus any arguments) to preexec
script (optional). This script will run immediately before each backup
operation (but not any rotations).
Full path (plus any arguments) to postexec
script (optional). This script will run immediately after each backup
operation (but not any rotations).
Paths to lvcreate, lvremove, mount and umount
commands, for use with Linux LVMs. The lvcreate, lvremove, mount and umount
commands are required for managing snapshots of LVM volumes and are otherwise
optional.
"name" refers to the name of this
backup level (e.g., hourly, daily, so also called the 'interval').
"number" is the number of snapshots for this type of interval that
will be retained. The value of "name" will be the command passed to
rsnapshot to perform this type of backup.
A deprecated alias for 'retain' is 'interval'.
Example: retain hourly 6
[root@localhost]# rsnapshot hourly
For this example, every time this is run, the following will happen:
<snapshot_root>/hourly.5/ will be deleted, if it exists.
<snapshot_root>/hourly.{1,2,3,4} will all be rotated +1, if they exist.
<snapshot_root>/hourly.0/ will be copied to
<snapshot_root>/hourly.1/ using hard links.
Each backup point (explained below) will then be rsynced to the corresponding
directories in <snapshot_root>/hourly.0/
Backup levels must be specified in the config file in order, from most frequent
to least frequent. The first entry is the one which will be synced with the
backup points. The subsequent backup levels (e.g., daily, weekly, etc) simply
rotate, with each higher backup level pulling from the one below it for its .0
directory.
Example:
retain hourly 6
retain daily 7
retain weekly 4
If your version of rsync supports --link-dest
(2.5.7 or newer), you can enable this to let rsync handle some things that GNU
cp or the built-in subroutines would otherwise do. Enabling this makes
rsnapshot take a slightly more complicated code branch, but it's the best way
to support special files on non-Linux systems.
sync_first changes the behaviour of rsnapshot.
When this is enabled, all calls to rsnapshot with various backup levels simply
rotate files. All backups are handled by calling rsnapshot with the
"sync" argument. The synced files are stored in a ".sync"
directory under the snapshot_root.
This allows better recovery in the event that rsnapshot is interrupted in the
middle of a sync operation, since the sync step and rotation steps are
separated. This also means that you can easily run "rsnapshot sync"
on the command line without fear of forcing all the other directories to
rotate up. This benefit comes at the cost of one more snapshot worth of disk
space. The default is 0 (off).
The amount of information to print out when
the program is run. Allowed values are 1 through 5. The default is 2.
1 Quiet Show fatal errors only 2 Default Show warnings and errors 3 Verbose Show equivalent shell commands being executed 4 Extra Verbose Same as verbose, but with more detail 5 Debug All kinds of information
This number means the same thing as
verbose above, but it determines how much data is written to the
logfile, if one is being written.
The only thing missing from this at the higher levels is the direct output from
rsync. We hope to add support for this in a future release.
Full filesystem path to the rsnapshot log
file. If this is defined, a log file will be written, with the amount of data
being controlled by loglevel. If this is commented out, no log file
will be written.
This gets passed directly to rsync using the
--include directive. This parameter can be specified as many times as needed,
with one pattern defined per line. See the rsync(1) man page for the
syntax.
This gets passed directly to rsync using the
--exclude directive. This parameter can be specified as many times as needed,
with one pattern defined per line. See the rsync(1) man page for the
syntax.
This gets passed directly to rsync using the
--include-from directive. See the rsync(1) man page for the
syntax.
This gets passed directly to rsync using the
--exclude-from directive. See the rsync(1) man page for the
syntax.
List of short arguments to pass to rsync. If
not specified, "-a" is the default. Please note that these must be
all next to each other. For example, "-az" is valid, while "-a
-z" is not.
"-a" is rsync's "archive mode" which tells it to copy as
much of the filesystem metadata as it can for each file. This specifically
does *not* include information about hard links, as that would greatly
increase rsync's memory usage and slow it down. If you need to preserve hard
links in your backups, then add "H" to this.
List of long arguments to pass to rsync.
Beginning with rsnapshot 1.2.0, this default has changed. In previous
versions, the default values were
Starting with version 1.2.0, the default values are
This directly affects how the destination paths in your backup points are
constructed. Depending on what behaviour you want, you can explicitly set the
values to make the program behave like the old version or the current version.
The newer settings are recommended if you're just starting. If you are
upgrading, read the upgrade guide in the INSTALL file in the source
distribution for more information.
Quotes are permitted in rsync_long_args, eg --rsync-path="sudo
/usr/bin/rsync". You may use either single (') or double (") quotes,
but nested quotes (including mixed nested quotes) are not permitted. Similar
quoting is also allowed in per-backup-point rsync_long_args.
--delete --numeric-ids
--delete --numeric-ids --relative --delete-excluded
Arguments to be passed to ssh. If not
specified, the default is none.
Arguments to be passed to du. If not
specified, the default is -csh. GNU du supports -csh, BSD du supports -csk,
Solaris du doesn't support -c at all. The GNU version is recommended, since it
offers the most features.
Lockfile to use when rsnapshot is run. This
prevents a second invocation from clobbering the first one. If not specified,
no lock file is used. Make sure to use a directory that is not world writeable
for security reasons. Use of a lock file is strongly recommended.
If a lockfile exists when rsnapshot starts, it will try to read the file and
stop with an error if it can't. If it *can* read the file, it sees if a
process exists with the PID noted in the file. If it does, rsnapshot stops
with an error message. If there is no process with that PID, then we assume
that the lockfile is stale and ignore it *unless* stop_on_stale_lockfile is
set to 1 in which case we stop.
stop_on_stale_lockfile defaults to 0.
Prevents rsync from crossing filesystem
partitions. Setting this to a value of 1 enables this feature. 0 turns it off.
This parameter is optional. The default is 0 (off).
Changes default behavior of rsnapshot and does
not initially remove the oldest snapshot. Instead it moves that directory to
_delete.[processid] and continues as normal. Once the backup has been
completed, the lockfile will be removed before rsnapshot starts deleting the
directory.
Enabling this means that snapshots get taken sooner (since the delete doesn't
come first), and any other rsnapshot processes are allowed to start while the
final delete is happening. This benefit comes at the cost of using more disk
space. The default is 0 (off).
The details of how this works have changed in rsnapshot version 1.3.1.
Originally you could only ever have one .delete directory per backup level.
Now you can have many, so if your next (eg) hourly backup kicks off while the
previous one is still doing a lazy delete you may temporarily have extra
_delete directories hanging around.
LVM snapshot(s) size (lvcreate --size
option).
Name to be used when creating the LVM logical
volume snapshot(s) (lvcreate --name option).
Path to the LVM Volume Groups.
Mount point to use to temporarily mount the
snapshot(s).
If you have used an older version of
rsnapshot, you might notice that the destination paths on the backup points
have changed. Please read the INSTALL file in the source distribution for
upgrade options.
Examples:
backup /etc/ localhost/
Backs up /etc/ to
<snapshot_root>/<retain>.0/localhost/etc/ using rsync on the local
filesystem
Backs up /usr/local/ to
<snapshot_root>/<retain>.0/localhost/usr/local/ using rsync on the
local filesystem
Backs up root@example.com:/etc/ to
<snapshot_root>/<retain>.0/example.com/etc/ using rsync over
ssh
Backs up root@example.com:/usr/local/ to
<snapshot_root>/<retain>.0/example.com/usr/local/ using rsync over
ssh
Backs up rsync://example.com/pub/ to
<snapshot_root>/<retain>.0/example.com/pub/ using an anonymous
rsync server. Please note that unlike backing up local paths and using rsync
over ssh, rsync servers have "modules", which are top level
directories that are exported. Therefore, the module should also be specified
in the destination path, as shown in the example above (the pub/ directory at
the end).
This is the same as the other examples, but
notice the fourth column. This is how you specify per-backup-point options to
over-ride global settings. This extra parameter can take several options,
separated by commas.
It is most useful when specifying per-backup rsync excludes thus:
backup root@somehost:/ somehost +rsync_long_args=--exclude=/var/spool/
Note the + sign. That tells rsnapshot to add to the list of arguments to
pass to rsync instead of replacing the list.
Backs up the LVM logical volume called home,
of volume group vg0, to <snapshot_root>/<interval>.0/lvm-vg0/.
Will create, mount, backup, unmount and remove an LVM snapshot for each lvm://
entry.
In this example, we specify a script or
program to run. This script should simply create files and/or directories in
its current working directory. rsnapshot will then take that output and move
it into the directory specified in the third column.
Please note that whatever is in the destination directory will be completely
deleted and recreated. For this reason, rsnapshot prevents you from specifying
a destination directory for a backup_script that will clobber other backups.
So in this example, say the backup_database.sh script simply runs a command
like:
#!/bin/sh
mysqldump -uusername mydatabase > mydatabase.sql
chmod u=r,go= mydatabase.sql # r-------- (0400)
# THIS IS A COMMENT, REMEMBER TABS MUST SEPARATE ALL ELEMENTS config_version 1.2 snapshot_root /.snapshots/ cmd_rsync /usr/bin/rsync cmd_ssh /usr/bin/ssh #cmd_cp /bin/cp cmd_rm /bin/rm cmd_logger /usr/bin/logger cmd_du /usr/bin/du linux_lvm_cmd_lvcreate /sbin/lvcreate linux_lvm_cmd_lvremove /sbin/lvremove linux_lvm_cmd_mount /bin/mount linux_lvm_cmd_umount /bin/umount linux_lvm_snapshotsize 2G linux_lvm_snapshotname rsnapshot linux_lvm_vgpath /dev linux_lvm_mountpath /mnt/lvm-snapshot retain hourly 6 retain daily 7 retain weekly 7 retain monthly 3 backup /etc/ localhost/ backup /home/ localhost/ backup_script /usr/local/bin/backup_mysql.sh mysql_backup/ backup root@foo.com:/etc/ foo.com/ backup root@foo.com:/home/ foo.com/ backup root@mail.foo.com:/home/ mail.foo.com/ backup rsync://example.com/pub/ example.com/pub/ backup lvm://vg0/xen-home/ lvm-vg0/xen-home/
USAGE¶
rsnapshot can be used by any user, but for system-wide backups you will probably want to run it as root. Since backups usually get neglected if human intervention is required, the preferred way is to run it from cron. When you are first setting up your backups, you will probably also want to run it from the command line once or twice to get a feel for what it's doing. Here is an example crontab entry, assuming that backup levels hourly, daily, weekly and monthly have been defined in /etc/rsnapshot.conf0 */4 * * * /usr/local/bin/rsnapshot
hourly
50 23 * * * /usr/local/bin/rsnapshot daily
40 23 * * 6 /usr/local/bin/rsnapshot weekly
30 23 1 * * /usr/local/bin/rsnapshot monthly
This example will do the following:
6 hourly backups a day (once every 4 hours, at
0,4,8,12,16,20)
1 daily backup every day, at 11:50PM
1 weekly backup every week, at 11:40PM, on Saturdays (6th day of week)
1 monthly backup every month, at 11:30PM on the 1st day of the month
It is usually a good idea to schedule the larger backup levels to run a bit
before the lower ones. For example, in the crontab above, notice that
"daily" runs 10 minutes before "hourly". The main reason
for this is that the daily rotate will pull out the oldest hourly and make
that the youngest daily (which means that the next hourly rotate will not need
to delete the oldest hourly), which is more efficient. A secondary reason is
that it is harder to predict how long the lowest backup level will take, since
it needs to actually do an rsync of the source as well as the rotate that all
backups do.
If rsnapshot takes longer than 10 minutes to do the "daily" rotate
(which usually includes deleting the oldest daily snapshot), then you should
increase the time between the backup levels. Otherwise (assuming you have set
the lockfile parameter, as is recommended) your hourly snapshot will
fail sometimes because the daily still has the lock.
Remember that these are just the times that the program runs. To set the number
of backups stored, set the retain numbers in /etc/rsnapshot.conf
To check the disk space used by rsnapshot, you can call it with the
"du" argument.
For example:
rsnapshot du
This will show you exactly how much disk space is taken up in the snapshot root.
This feature requires the UNIX du command to be installed on your
system, for it to support the "-csh" command line arguments, and to
be in your path. You can also override your path settings and the flags passed
to du using the cmd_du and du_args parameters.
It is also possible to pass a relative file path as a second argument, to get a
report on a particular file or subdirectory.
rsnapshot du localhost/home/
The GNU version of "du" is preferred. The BSD version works well also,
but does not support the -h flag (use -k instead, to see the totals in
kilobytes). Other versions of "du", such as Solaris, may not work at
all.
To check the differences between two directories, call rsnapshot with the
"diff" argument, followed by two backup levels or directory paths.
For example:
rsnapshot diff daily.0 daily.1
rsnapshot diff daily.0/localhost/etc daily.1/localhost/etc
rsnapshot diff /.snapshots/daily.0 /.snapshots/daily.1
This will call the rsnapshot-diff program, which will scan both directories
looking for differences (based on hard links).
rsnapshot sync
When sync_first is enabled, rsnapshot
must first be called with the sync argument, followed by the other
usual cron entries. The sync should happen as the lowest, most frequent backup
level, and right before. For example:
0 */4 * * * /usr/local/bin/rsnapshot sync
&& /usr/local/bin/rsnapshot hourly
50 23 * * * /usr/local/bin/rsnapshot daily
40 23 1,8,15,22 * * /usr/local/bin/rsnapshot weekly
30 23 1 * * /usr/local/bin/rsnapshot monthly
When sync_first is enabled, all sync
behaviour happens during an additional sync step (see above). When using the
sync argument, it is also possible to specify a backup point destination as an
optional parameter. If this is done, only backup points sharing that
destination path will be synced.
For example, let's say that example.com is a destination path shared by one or
more of your backup points.
rsnapshot sync example.com
EXIT VALUES¶
0 All operations completed successfully
1 A fatal error occurred
2 Some warnings occurred, but the backup still finished
FILES¶
/etc/rsnapshot.confSEE ALSO¶
rsync(1), ssh(1), logger(1), sshd(1), ssh-keygen(1), perl(1), cp(1), du(1), crontab(1)DIAGNOSTICS¶
Use the -t flag to see what commands would have been executed. This will show you the commands rsnapshot would try to run. There are a few minor differences (for example, not showing an attempt to remove the lockfile because it wasn't really created in the test), but should give you a very good idea what will happen. Using the -v, -V, and -D flags will print increasingly more information to STDOUT. Make sure you don't have spaces in the config file that you think are actually tabs. Much other weird behavior can probably be attributed to plain old file system permissions and ssh authentication issues.BUGS¶
Please report bugs (and other comments) to the rsnapshot-discuss mailing list: http://lists.sourceforge.net/lists/listinfo/rsnapshot-discussNOTES¶
Make sure your /etc/rsnapshot.conf file has all elements separated by tabs. See /usr/share/doc/rsnapshot/examples/rsnapshot.conf.default.gz for a working example file. Make sure you put a trailing slash on the end of all directory references. If you don't, you may have extra directories created in your snapshots. For more information on how the trailing slash is handled, see the rsync (1) manpage. Make sure to make the snapshot directory chmod 700 and owned by root (assuming backups are made by the root user). If the snapshot directory is readable by other users, they will be able to modify the snapshots containing their files, thus destroying the integrity of the snapshots. If you would like regular users to be able to restore their own backups, there are a number of ways this can be accomplished. One such scenario would be: Set snapshot_root to /.private/.snapshots in /etc/rsnapshot.conf Set the file permissions on these directories as follows:drwx------ /.private
drwxr-xr-x /.private/.snapshots
Export the /.private/.snapshots directory over read-only NFS, a read-only Samba
share, etc.
See the rsnapshot HOWTO for more information on making backups accessible to
non-privileged users.
For ssh to work unattended through cron, you will probably want to use public
key logins. Create an ssh key with no passphrase for root, and install the
public key on each machine you want to backup. If you are backing up system
files from remote machines, this probably means unattended root logins.
Another possibility is to create a second user on the machine just for
backups. Give the user a different name such as "rsnapshot", but
keep the UID and GID set to 0, to give root privileges. However, make logins
more restrictive, either through ssh configuration, or using an alternate
shell.
BE CAREFUL! If the private key is obtained by an attacker, they will have free
run of all the systems involved. If you are unclear on how to do this, see
ssh(1), sshd(1), and
ssh-keygen(1).
Backup scripts are run as the same user that rsnapshot is running as. Typically
this is root. Make sure that all of your backup scripts are only writable by
root, and that they don't call any other programs that aren't owned by root.
If you fail to do this, anyone who can write to the backup script or any
program it calls can fully take over the machine. Of course, this is not a
situation unique to rsnapshot.
By default, rsync transfers are done using the --numeric-ids option. This means
that user names and group names are ignored during transfers, but the UID/GID
information is kept intact. The assumption is that the backups will be
restored in the same environment they came from. Without this option,
restoring backups for multiple heterogeneous servers would be unmanageable. If
you are archiving snapshots with GNU tar, you may want to use the
--numeric-owner parameter. Also, keep a copy of the archived system's
/etc/passwd and /etc/group files handy for the UID/GID to name mapping.
If you remove backup points in the config file, the previously archived files
under those points will permanently stay in the snapshots directory unless you
remove the files yourself. If you want to conserve disk space, you will need
to go into the <snapshot_root> directory and manually remove the files
from the smallest backup level's ".0" directory.
For example, if you were previously backing up /home/ with a destination of
localhost/, and hourly is your smallest backup level, you would need to do the
following to reclaim that disk space:
rm -rf
<snapshot_root>/hourly.0/localhost/home/
Please note that the other snapshots previously made of /home/ will still be
using that disk space, but since the files are flushed out of hourly.0/, they
will no longer be copied to the subsequent directories, and will thus be
removed in due time as the rotations happen.
AUTHORS¶
Mike Rubel - http://www.mikerubel.org/computers/rsync_snapshots/- - Created the original shell scripts on which this project is based
- - Primary author and previous maintainer of rsnapshot.
- - Current co-maintainer of rsnapshot
- - Wrote the rsnapshot-diff utility
- - Improved how use_lazy_deletes work so slow deletes don't screw up the next backup at that backup level.
- - Co-maintainer, with responsibility for release management since 1.2.9
- - Fixed race condition in lock file creation, improved error reporting
- - Allowed remote ssh directory paths starting with "~/" as well as "/"
- - Fixed a number of other bugs and buglets
- - Created the RPM .spec file which allowed the RPM package to be built, among other things.
- - Added the one_fs feature, autoconf support, good advice, and much more.
- - Added and maintains the rsnapshot entry in the FreeBSD ports tree.
- - Provided access to his computer museum for software testing.
- - Found several subtle bugs and provided fixes for them.
- - Fixed a compatibility bug in utils/backup_smb_share.sh
- - Added (and previously maintained) the Debian rsnapshot package.
- - Improved the exclusion rules to avoid backing up the snapshot root (among other things).
- - Enhanced error reporting to include command line options.
- - Fixed typos in program and man page
Added cwrsync permanent-share support
Robert Jackson ( RobertJ@promedicalinc.com)
Added use_lazy_deletes feature
Justin Grote ( justin@grote.name)
Improved rsync error reporting code
Anthony Ettinger ( apwebdesign@yahoo.com)
Wrote the utils/mysqlbackup.pl script
Sherman Boyd
Wrote utils/random_file_verify.sh script
William Bear ( bear@umn.edu)
Wrote the utils/rsnapreport.pl script (pretty
summary of rsync stats)
Eric Anderson ( anderson@centtech.com)
Improvements to utils/rsnapreport.pl.
Alan Batie ( alan@batie.org)
Bug fixes for include_conf
Dieter Bloms ( dieter@bloms.de)
Multi-line configuration options
Henning Moll ( newsScott@gmx.de)
stop_on_stale_lockfile
Ben Low ( ben@bdlow.net)
Linux LVM snapshot support
COPYRIGHT¶
Copyright (C) 2003-2005 Nathan Rosenquist Portions Copyright (C) 2002-2007 Mike Rubel, Carl Wilhelm Soderstrom, Ted Zlatanov, Carl Boe, Shane Liebling, Bharat Mediratta, Peter Palfrader, Nicolas Kaiser, David Cantrell, Chris Petersen, Robert Jackson, Justin Grote, David Keegel, Alan Batie, Dieter Bloms, Henning Moll, Ben Low, Anthony Ettinger This man page is distributed under the same license as rsnapshot: the GPL (see below). This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version. This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details. You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA2011-07-09 |