'\" t
.\"     Title: pegasus-transfer
.\"    Author: [see the "Author" section]
.\" Generator: DocBook XSL Stylesheets v1.79.1 <http://docbook.sf.net/>
.\"      Date: 11/09/2018
.\"    Manual: Pegasus Manual
.\"    Source: Pegasus 4.4.0
.\"  Language: English
.\"
.TH "PEGASUS\-TRANSFER" "1" "11/09/2018" "Pegasus 4\&.4\&.0" "Pegasus Manual"
.\" -----------------------------------------------------------------
.\" * Define some portability stuff
.\" -----------------------------------------------------------------
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.\" http://bugs.debian.org/507673
.\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html
.\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\" -----------------------------------------------------------------
.\" * set default formatting
.\" -----------------------------------------------------------------
.\" disable hyphenation
.nh
.\" disable justification (adjust text to left margin only)
.ad l
.\" -----------------------------------------------------------------
.\" * MAIN CONTENT STARTS HERE *
.\" -----------------------------------------------------------------
.SH "NAME"
pegasus-transfer \- Handles data transfers in Pegasus workflows\&.
.SH "SYNOPSIS"
.sp
.nf
\fBpegasus\-transfer\fR [\fB\-h\fR]
                   [\fB\-\-file\fR \fIinputfile\fR]
                   [\fB\-\-threads\fR \fInumber_threads\fR]
                   [\fB\-\-max\-attempts\fR \fIattempts\fR]
                   [\fB\-\-debug\fR]
.fi
.SH "DESCRIPTION"
.sp
\fBpegasus\-transfer\fR takes a list of url pairs, either on stdin or with an input file, determines the correct tool to use for the transfer and executes the transfer\&. Some of the protocols pegasus\-transfer can handle are GridFTP, SRM, Amazon S3, HTTP, and local cp/symlinking\&. Failed transfers are retried\&.
.sp
Note that pegasus\-transfer is a tool mostly used internally in Pegasus workflows, but the tool can be used stand alone as well\&.
.SH "OPTIONS"
.PP
\fB\-h\fR, \fB\-\-help\fR
.RS 4
Prints a usage summary with all the available command\-line options\&.
.RE
.PP
\fB\-f\fR \fIinputfile\fR, \fB\-\-file\fR \fIinputfile\fR
.RS 4
File with input pairs\&. If not given, stdin will be used\&.
.RE
.PP
\fB\-m\fR, \fB\-\-max\-attempts\fR \fIattempts\fR
.RS 4
Maximum number of attempts for retrying failed transfers\&.
.RE
.PP
\fB\-t\fR, \fB\-\-threads\fR \fInumber_threads\fR
.RS 4
The number of threads to use\&. This controls the parallelism of transfers\&.
.RE
.PP
\fB\-d\fR, \fB\-\-debug\fR
.RS 4
Enables debugging output\&.
.RE
.SH "EXAMPLE"
.sp
.if n \{\
.RS 4
.\}
.nf
$ pegasus\-transfer
# src 1 local_site
file:///etc/hosts
# dst 1 local_site
file:///tmp/foo
CTRL+D
.fi
.if n \{\
.RE
.\}
.SH "CREDENTIAL HANDLING"
.sp
Credentials used for transfers can be specified with a combination of comments in the input file format and environment variables\&. For example, give the following input file:
.sp
.if n \{\
.RS 4
.\}
.nf
# src 1 isi
gsiftp://workflow\&.isi\&.edu/data/file\&.dat
# dst 1 tacc_stampede
gsiftp://gridftp\&.stampede\&.tacc\&.utexas\&.edu/scratch/file\&.dat
.fi
.if n \{\
.RE
.\}
.sp
pegasus\-transfer will expect either one environment variable specifying one credential to be used on both end of the connection (X509_USER_PROXY), or two separate environment variables specifying two different credentials to be used on the two ends of the connection\&. The the latter case, the environment variables are derived from the site names provided in the input file commments\&. In the example above, the environment variables would be named X509_USER_PROXY_isi and X509_USER_PROXY_tacc_stampede
.SH "THREADING"
.sp
In order to speed up data transfers, pegasus\-transfer will start a set of transfers in parallel using threads\&. Threads are turned off when retrying failed transfers\&.
.SH "AUTHOR"
.sp
Pegasus Team \m[blue]\fBhttp://pegasus\&.isi\&.edu\fR\m[]