Scroll to navigation

cback3-amazons3-sync(1) Kenneth J. Pronovici cback3-amazons3-sync(1)

NAME

cback3-amazons3-sync - Synchronize a local directory with an Amazon S3 bucket

SYNOPSIS

cback3-amazons3-sync [switches] sourceDir s3BucketUrl

DESCRIPTION

This is the Cedar Backup 3 Amazon S3 sync tool. It synchronizes a local directory to an Amazon S3 cloud storage bucket. After the sync is complete, a validation step is taken. An error is reported if the contents of the bucket do not match the source directory, or if the indicated size for any file differs.

Generally, one can run the cback3-amazons3-sync command with no special switches. This will start it using the default Cedar Backup log file, etc. You only need to use the switches if you need to change the default behavior.

MIGRATING FROM VERSION 2 TO VERSION 3

The main difference between Cedar Backup version 2 and Cedar Backup version 3 is the targeted Python interpreter. For most users, migration should be straightforward. See the discussion found at cback3(1) or reference the Cedar Backup user guide.

ARGUMENTS

The source directory on a local disk.
The URL specifying the location of the Amazon S3 cloud storage bucket to synchronize with, like s3://example.com-backup/subdir.

SWITCHES

Display usage/help listing.
Display version information.
Print verbose output to the screen as well writing to the logfile. When this option is enabled, most information that would normally be written to the logfile will also be written to the screen.
Specify the path to an alternate logfile. The default logfile file is /var/log/cback3.log.
Specify the ownership of the logfile, in the form user:group. The default ownership is root:adm, to match the Debian standard for most logfiles. This value will only be used when creating a new logfile. If the logfile already exists when the cback3 script is executed, it will retain its existing ownership and mode. Only user and group names may be used, not numeric uid and gid values.
Specify the permissions for the logfile, using the numeric mode as in chmod(1). The default mode is 640 (-rw-r-----). This value will only be used when creating a new logfile. If the logfile already exists when the cback3 script is executed, it will retain its existing ownership and mode.
Record some sub-command output to the logfile. When this option is enabled, all output from system commands will be logged. This might be useful for debugging or just for reference.
Write debugging information to the logfile. This option produces a high volume of output, and would generally only be needed when debugging a problem. This option implies the --output option, as well.
Dump a Python stack trace instead of swallowing exceptions. This forces Cedar Backup to dump the entire Python stack trace associated with an error, rather than just propagating last message it received back up to the user interface. Under some circumstances, this is useful information to include along with a bug report.
Display runtime diagnostic information and then exit. This diagnostic information is often useful when filing a bug report.
Only verify the S3 bucket contents against the directory on disk. Do not make any changes to the S3 bucket or transfer any files. This is intended as a quick check to see whether the sync is up-to-date. Although no files are transferred, the tool will still execute the source filename encoding check.
Implement a partial or "upload only" sync, instead of a full synchronization. Normally, synchronization would remove files that exist in S3 but do not exist in the directory on disk. With this flag, new files are uploaded, but no files are removed in S3.
The AWS CLI S3 sync process is very picky about filename encoding. Files that the Linux filesystem handles with no problems can cause problems in S3 if the filename cannot be encoded properly in your configured locale. As of this writing, filenames like this will cause the sync process to abort without transferring all files as expected. To avoid confusion, the tool tries to guess which files in the source directory will cause problems, and refuses to execute the AWS CLI S3 sync if any problematic files exist. If you'd rather proceed anyway, use this flag.

RETURN VALUES

This command returns 0 (zero) upon normal completion, and several other error codes related to particular errors.

1
The Python interpreter version is < 3.7.
2
Error processing command-line arguments.
3
Error configuring logging.
5
Backup was interrupted with a CTRL-C or similar.
6
Other error during processing.

NOTES

This tool is a wrapper over the Amazon AWS CLI interface found in the aws(1) command. Specifically, cback3-amazons3-sync invokes "aws s3 sync" followed by "aws s3api list-objects".

Cedar Backup itself is designed to run as root. However, cback3-amazons3-sync can be run safely as any user that is configured to use the Amazon AWS CLI interface. The aws(1) command will be executed by the same user which is executing cback3-amazons3-sync.

You must configure the AWS CLI interface to have a valid connection to Amazon S3 infrastructure before using cback3-amazons3-sync. For more information about how to accomplish this, see the Cedar Backup user guide.

SEE ALSO

cback3(1)

FILES

/var/log/cback3.log - Default log file

URLS

https://github.com/pronovic/cedar-backup3

BUGS

If you find a bug, please report it.

If possible, give me the output from --diagnostics, all of the error messages that the script printed into its log, and also any stack-traces (exceptions) that Python printed. It would be even better if you could tell me how to reproduce the problem, for instance by sending me your configuration file.

Report bugs to <support@cedar-solutions.com> or via GitHub issues tracker.

AUTHOR

Written and maintained by Kenneth J. Pronovici <pronovic@ieee.org> with contributions from others.

COPYRIGHT

Copyright (c) 2004-2020 Kenneth J. Pronovici.

This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Nov 2020 Cedar Backup 3