sitecopy(1) | User Manuals | sitecopy(1) |
NAME¶
sitecopy - maintain remote copies of web sitesSYNOPSIS¶
sitecopy [options] [operation mode] sitename ...DESCRIPTION¶
sitecopy is for copying locally stored web sites to remote web servers. A single command will upload files to the server which have changed locally, and delete files from the server which have been removed locally, to keep the remote site synchronized with the local site. The aim is to remove the hassle of uploading and deleting individual files using an FTP client. sitecopy will also optionally try to spot files you move locally, and move them remotely.GETTING STARTED¶
This section covers how to start maintaining a web site using sitecopy. After introducing the basics, two situations are covered: first, where you have already upload the site to the remote server; second, where you haven't. Lastly, normal site maintenance activities are explained.Introducing the Basics¶
If you have not already done so, you need to create an rcfile, which will store information about the sites you wish to administer. You also need to create a storage directory, which sitecopy uses to record the state of the files on each of the remote sites. The rcfile and storage directory must both be accessible only by you - sitecopy will not run otherwise. To create the storage directory with the correct permissions, use the commandExisting Remote Site¶
If you have already uploaded the site to the remote server, ensure your local files are synchronized with the remote files. Then, runNew Remote Site¶
Ensure that the root directory of the site has been created on the server by the server administrator. RunSite Maintenance¶
After setting up the site as given in one of the two above sections, you can now start editing your local files as normal. When you have finished a set of changes, and you want to update the remote copy of the site, run:Synchronization Problems¶
In some circumstances, the actual files which make up the remote site will be different from what sitecopy thinks is on the remote site. This can happen, for instance, if the connection to the server is broken during an update. When this situation arises, Fetch Mode should be used to fetch the list of files making up the site from the remote server.INVOCATION¶
In normal operation, specify a single operation mode, followed by any options you choose, then one or more site names. For instance,OPERATION MODES¶
- -l, --list
- List Mode - produces a listing of all the differences between the local files and the remote copy for the specified sites.
- -ll, --flatlist
- Flat list Mode - like list mode, except the output produced is suitable for parsing by an external script or program. An AWK script, changes.awk. is provided which produces an HTML page from this mode.
- -u, --update
- Update Mode - updates the remote copy of the specified sites.
- -f, --fetch
- Fetch Mode - fetches the list of files from the remote server. Note that this mode has only limited support in FTP - the server must accept the MDTM command, and use a Unix-style 'ls' for LIST implementation.
- -s, --synchronize
- Synchronize Mode - updates the local site from the remote copy. WARNING: This mode overwrites local files. Use with care.
- -i, --initialize
- Initialization Mode - initializes the sites specified - making sitecopy think there are NO files on the remote server.
- -c, --catchup
- Catchup Mode - makes sitecopy think the local site is exactly the same as the remote copy.
- -v, --view
- View Mode - displays all the site definitions from the rcfile.
- -e, --verify
-
Verify stored state of site matches real remote state
- -h, --help
- Display help information.
- -V, --version
- Display version information.
OPTIONS¶
- -y, --prompting
- -g, --logfile=FILE
-
Append debugging messages to FILE (else use stderr)
- -x, --create-remote
-
Create root for remote site
- -n, --dry-run
-
Display but do not carry out the operation Applicable in Update Mode only, will prompt the user for confirmation for each update (i.e., creating a directory, uploading a file etc.).
- -r RCFILE, --rcfile=RCFILE
- Specify an alternate run control file location.
- -p PATH, --storepath=PATH
- Specify an alternate location to use for the remote site storage directory.
- -q, --quiet
- Quiet output - display the filename only for each update performed.
- -qq, --silent
- Very quiet output - display nothing for each update performed.
- -o, --show-progress
- Applicable in Update Mode only, displays the progress (percentage complete) of data transfer.
- -k, --keep-going
- Keep going past errors in Update Mode or Synch
Mode
- -a, --allsites
- Perform the given operation on all sites - applicable for all modes except View Mode, for which it has no effect.
- -d MASK, --debug=KEY[,KEY...]
- Turns on debugging. A list of comma-separated keywords
should be given. Each keyword may be one of:
socket Socket handling
files File handling
rcfile rcfile parser
http HTTP driver
httpbody Display response bodies in HTTP
ftp FTP driver
sftp SFTP driver
xml XML parsing information
xmlparse Low-level XML parsing information
httpauth HTTP authentication information
cleartext Display passwords in plain text
CONCEPTS¶
The stored state of a site is the snapshot of the state of the site saved into the storage directory (~/.sitecopy/). The storage file is used to record this state between invocations. In update mode, sitecopy builds up a files list for each site by scanning the local directory, reading in the stored state, and comparing the two - determining which files have changed, which have moved, and so on.CONFIGURATION¶
Configuration is performed via the run control file (rcfile). This file contains a set of site definitions. A unique name is assigned to every site definition, which is used on the command line to refer to the site. Each site definition contains the details of the server the site is stored on, how the site may be accessed at that server, where the site is held locally and remotely, and any other options for the site.Site Definition¶
A site definition is made up of a series of lines:[ port port-number ]
[ username username ]
[ password password ]
[ proxy-server proxy-name
[ url siteURL ]
[ protocol { ftp | sftp | webdav } ]
[ ftp nopasv ]
[ ftp showquit ]
[ ftp { usecwd | nousecwd } ]
[ http expect ]
[ http secure ]
[ safe ]
[ state { checksum | timesize } ]
[ permissions { ignore | exec | all | dir } ]
[ symlinks { ignore | follow | maintain } ]
[ nodelete ]
[ nooverwrite ]
[ checkmoved [renames] ]
[ tempupload ]
[ exclude pattern ]...
[ ignore pattern ]...
[ ascii pattern ]...
Remote Server Options¶
The server key is used to specify the remote server the site is stored on. This may be either a DNS name or IP address. A connection is made to the default port for the protocol used, or that given by the port key. sitecopy supports the WebDAV or (S)FTP protocols - the protocol key specifies which to use, taking the value of either webdav or ftp/sftp respectively. By default, FTP will be used.File State¶
File state is stored in the storage files (~/.sitecopy/*), and is used to discover when a file has been changed. Two methods are supported, and can be selected using the state option, with either parameter: timesize (the default), and checksum.Safe Mode¶
Safe Mode is enabled by using the safe key. When enabled, each time a file is uploaded to the server, the modification time of the file as on the server is recorded. Subsequently, whenever this file has been changed locally and is to be uploaded again, the current modification time of the file on the server is retrieved, and compared with the stored value. If these differ, then the remote copy of the file has been altered by a foreign party. A warning message is issued, and your local copy of the file will not be uploaded over it, to prevent losing any changes.File Storage Locations¶
The remote key specifies the root directory of the remote copy of the site. It may be in the form of an absolute pathname, e.g.File Permissions Handling¶
File permissions handling is dictated by the permissions key, which may be given one of three values:- ignore
- to ignore file permissions completely (the default),
- exec
- to mirror the permissions of executable files only,
- all
- to mirror the permissions of all files.
Symbolic Link Handling¶
Symlinks found in the local site can be either ignored, followed, or maintained. In 'follow' mode, the files references by the symlinks will be uploaded in their place. In 'maintain' mode, the link will be created remotely as well, see below for more information. The mode used for each site is specified with the symlinks rcfile key, which may take the value of ignore, follow or maintain to select the mode as appropriate.Symbolic link Maintain Mode¶
This mode is currently only supported by the WebDAV driver, and will work only with servers which implement WebDAV Advanced Collections, which is a work-in-progress. The target of the link on the server is literally copied from the target of the symlink. Hint: you can use URL's if you like:Deleting and Moving Remote Files¶
The nodelete option may be used to prevent remote files from ever being deleted. This may be useful if you keep large amounts of data on the remote server which you do not need to store locally as well.Excluding Files¶
Files may be excluded from the files list by use of the exclude key, which accepts shell-style globbing patterns. For example, useIgnoring Local Changes to Files¶
The ignore option is used to instruct sitecopy to ignore any local changes made to a file. If a change is made to the contents of an ignored file, this file will not be uploaded by update mode. Ignored files will be created, moved and deleted as normal.FTP Transfer Mode¶
To specify the FTP transfer mode for files, use the ascii key. Any files which are transferred using ASCII mode have CRLF/LF translation performed appropriately. For example, useRETURN VALUES¶
Return values are specified for different operation modes. If multiple sites are specified on the command line, the return value is in respect to the last site given.Update Mode¶
-1 ... update never even started - configuration problem
0 ... update was entirely successful.
1 ... update went wrong somewhere
2 ... could not connect or login to server
List Mode (default mode of operation)¶
-1 ... could not form list - configuration problem
0 ... the remote site does not need updating
1 ... the remote site needs updating
EXAMPLE RCFILE CONTENTS¶
FTP Server, Simple Usage¶
Fred's site is uploaded to the FTP server 'my.server.com' and held in the directory 'public_html', which is in the login directory. The site is stored locally in the directory /home/fred/html.server my.server.com
url http://www.server.com/fred
username fred
password juniper
local /home/fred/html/
remote ~/public_html/
FTP Server, Complex Usage¶
Here, Freda's site is uploaded to the FTP server ´ftp.elsewhere.com´, where it is held in the directory /www/freda/. The local site is stored in /home/freda/sites/elsewhere/server ftp.elsewhere.com
username freda
password blahblahblah
local /home/freda/sites/elsewhere/
remote /www/freda/
# Freda wants files with a .bak extension or a
# trailing ~ to be ignored:
exclude *.bak
exclude *~
WebDAV Server, Simple Usage¶
This example shows use of a WebDAV server.server dav.wow.com
protocol webdav
username pow
password zap
local /home/joe/www/super/
remote /
FILES¶
~/.sitecopyrc Default run control file location.BUGS¶
Known problems: Fetch + synch modes are NOT reliable for FTP. If you need reliable operation of fetch or synch modes, you shouldn't be using sitecopy. Try rsync instead.SEE ALSO¶
rsync(1), ftp(1), sftp(1), mirror(1)STANDARDS¶
[Listed for reference only, no claim of compliance to any of the below standards is made.]DRAFT STANDARDS¶
draft-ietf-ftpext-mlst-05.txt - Extensions to FTPAUTHOR¶
Joe Orton and others.June 2001 | sitecopy |