NAME¶

parsyncfp2 - a MultiHost parallel rsync wrapper

SYNOPSIS¶

parsyncfp2 [OPTION]... SRC... DEST

DESCRIPTION¶

parsyncfp2 wraps rsync to provide load balancing and parallel operation across network connections to substantially increase the amount of data it can send simultaneously. parsyncfp2 exploits parallel operation to decrease the impact of the TCP Round Trip Time(rtt) to significantly increase the total bandwidth of data across networks. Even on low-latency networks, it can speed large transfers by 4-10x. However, it is not effective for small transfers, since the startup overhead will slow the effective throughput.

parsyncfp2 uses fpart to split the source data into chunks of a specified size (default: 10 GB), and then starts parallel instances of rsync to transfer those chunks. parsyncfp2 only needs to be installed on the source machine. It can optionally monitor the system’s load average and limit the impact of the data transfer. A multi-host mode is also available (several senders and receivers).

MAIN OPTIONS¶

--np|NP=processes

--chunksize|cs=size

--ro=options

--help

IMPORTANT NOTES¶

rsync --delete options will not work with --ro because the multiple parallel rsyncs that parsyncfp launches are independent and therefore don’t know about each other (and so cannot exchange info about what should be deleted or not. Use a final, separate rsync --delete to clean up the transfer if that’s your need.

parsyncfp2 only works on dirs and files that originate from the current dir (or specified via "--startdir"). You cannot include dirs and files from discontinuous or higher-level dirs.

EXAMPLE¶

Use 4 rsync processes and a chunksize of 1G, and change rsync options to enable compression

ADDITIONAL DOCUMENTATION¶

The parsyncfp2 manual