'\" t .\" Title: pegasus-statistics .\" Author: [see the "Authors" section] .\" Generator: DocBook XSL Stylesheets v1.79.1 .\" Date: 11/09/2018 .\" Manual: Pegasus Manual .\" Source: Pegasus 4.4.0 .\" Language: English .\" .TH "PEGASUS\-STATISTICS" "1" "11/09/2018" "Pegasus 4\&.4\&.0" "Pegasus Manual" .\" ----------------------------------------------------------------- .\" * Define some portability stuff .\" ----------------------------------------------------------------- .\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .\" http://bugs.debian.org/507673 .\" http://lists.gnu.org/archive/html/groff/2009-02/msg00013.html .\" ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" ----------------------------------------------------------------- .\" * set default formatting .\" ----------------------------------------------------------------- .\" disable hyphenation .nh .\" disable justification (adjust text to left margin only) .ad l .\" ----------------------------------------------------------------- .\" * MAIN CONTENT STARTS HERE * .\" ----------------------------------------------------------------- .SH "NAME" pegasus-statistics \- A tool to generate statistics about the workflow run\&. .SH "SYNOPSIS" .sp .nf \fBpegasus\-statistics\fR [\fB\-h\fR|\fB\-\-help\fR] [\fB\-o\fR|\fB\-\-output\fR \fIdir\fR] [\fB\-c\fR|\fB\-\-conf\fR \fIpropfile\fR] [\fB\-p\fR|\fB\-\-statistics\-level\fR \fIlevel\fR] [\fB\-t\fR|\fB\-\-time\-filter\fR \fIfilter\fR] [\fB\-i\fR|\fB\-\-ignore\-db\-inconsistency\fR] [\fB\-v\fR|\fB\-\-verbose\fR] [\fB\-q\fR|\fB\-\-quiet\fR] [\fB\-m\fR|\fB\-\-multiple\-wf\fR] [\fB\-p\fR|\fB\-\-ispmc\fR] [\fB\-u\fR|\fB\-\-isuuid\fR] [[\fIsubmitdir \&.\&.\fR] | [\fIworkflow_uuid \&.\&.\fR]] .fi .SH "DESCRIPTION" .sp pegasus\-statistics generates statistics about the workflow run like total jobs/tasks/sub workflows ran, how many succeeded/failed etc\&. It generates job instance statistics like run time, condor queue delay etc\&. It generates invocation statistics information grouped by transformation name\&. It also generates job instance and invocation statistics information grouped by time and host\&. .SH "OPTIONS" .PP \fB\-h\fR, \fB\-\-help\fR .RS 4 Prints a usage summary with all the available command\-line options\&. .RE .PP \fB\-o\fR \fIdir\fR, \fB\-\-output\fR \fIdir\fR .RS 4 Writes the output to the given directory\&. .RE .PP \fB\-c\fR \fIpropfile\fR, \fB\-\-conf\fR \fIpropfile\fR .RS 4 The properties file to use\&. This option overrides all other property files\&. .RE .PP \fB\-s\fR \fIlevel\fR, \fB\-\-statistics\-level\fR \fIlevel\fR .RS 4 Specifies the statistics information to generate\&. Valid levels are: \fBall\fR, \fBsummary\fR, \fBwf_stats\fR, \fBjb_stats\fR, \fBtf_stats\fR, and \fBti_stats\fR\&. Default is \fBsummary\fR\&. The output generated by pegasus\-statistics is based on the the \fIlevel\fR set: .sp .RS 4 .ie n \{\ \h'-04'\(bu\h'+03'\c .\} .el \{\ .sp -1 .IP \(bu 2.3 .\} \fBall\fR: generates all the statistics information\&. .RE .sp .RS 4 .ie n \{\ \h'-04'\(bu\h'+03'\c .\} .el \{\ .sp -1 .IP \(bu 2.3 .\} \fBsummary\fR: generates the workflow statistics summary\&. In the case of a hierarchical workflow the summary is across all sub workflows\&. .RE .sp .RS 4 .ie n \{\ \h'-04'\(bu\h'+03'\c .\} .el \{\ .sp -1 .IP \(bu 2.3 .\} \fBwf_stats\fR: generates the workflow statistics information of each individual workflow\&. In case of a hierarchical workflow the workflow statistics are created for each sub workflow\&. .RE .sp .RS 4 .ie n \{\ \h'-04'\(bu\h'+03'\c .\} .el \{\ .sp -1 .IP \(bu 2.3 .\} \fBjb_stats\fR: generates the job statistics information of each individual workflow\&. In case of hierarchical workflow the job statistics is created for each sub workflows\&. Note: Not supported when generating statistics over multiple workflows\&. .RE .sp .RS 4 .ie n \{\ \h'-04'\(bu\h'+03'\c .\} .el \{\ .sp -1 .IP \(bu 2.3 .\} \fBtf_stats\fR: generates the invocation statistics information of each individual workflow grouped by transformation name \&.In case of hierarchical workflow the transformation statistics is created for each sub workflows\&. .RE .sp .RS 4 .ie n \{\ \h'-04'\(bu\h'+03'\c .\} .el \{\ .sp -1 .IP \(bu 2.3 .\} \fBti_stats\fR: generates the job instance and invocation statistics like total count and runtime grouped by time and host\&. .RE .RE .PP \fB\-t\fR \fIfilter\fR, \fB\-\-time\-filter\fR \fIfilter\fR .RS 4 Specifies the time filter to group the time statistics\&. Valid \fIfilter\fR values are: \fBmonth\fR, \fBweek\fR, \fBday\fR, \fBhour\fR\&. Default is \fBday\fR\&. .RE .PP \fB\-i\fR, \fB\-\-ignore\-db\-inconsistency\fR .RS 4 Turn off the the check for database consistency\&. .RE .PP \fB\-v\fR, \fB\-\-verbose\fR .RS 4 Increases the log level\&. If omitted, the default level will be set to WARNING\&. When this option is given, the log level is changed to INFO\&. If this option is repeated, the log level will be changed to DEBUG\&. .RE .PP \fB\-q\fR, \fB\-\-quiet\fR .RS 4 Decreases the log level\&. If omitted, the default level will be set to WARNING\&. When this option is given, the log level is changed to ERROR\&. .RE .PP \fB\-m\fR, \fB\-\-multiple\-wf\fR .RS 4 Set this option when generating statistics over more than one workflow\&. The tool automatically sets this flag if multiple submit directories or multiple workflow UUIDs are provided\&. This option would need to be set explicitly only to generate statistics over all workflows in a single STAMPEDE database\&. NOTE: When workflows are specified as UUIDs the \-\-conf options needs to be set for the tool to determine the STAMPEDE database URL\&. .RE .PP \fB\-p\fR, \fB\-\-ispmc\fR .RS 4 Set this flag to generate statistics for workflows which are run with PMC clustering enabled\&. It is recommended that this option be used when calculating statistics over multiple workflow runs\&. .RE .PP \fB\-u\fR, \fB\-\-isuuid\fR .RS 4 Set this option if the positional argument are workflow UUIDs\&. NOTE: When workflows are specified as UUIDs the \-\-conf options needs to be set for the tool to determine the STAMPEDE database URL\&. .RE .SH "EXAMPLE" .sp Runs pegasus\-statistics and writes the output to the given directory: .sp .if n \{\ .RS 4 .\} .nf $ pegasus\-statistics \-o /scratch/statistics /scratch/grid\-setup/run0001 .fi .if n \{\ .RE .\} .sp Runs pegasus\-statistics over a workflow run identified by a single workflow UUID: .sp .if n \{\ .RS 4 .\} .nf $ pegasus\-statistics \-\-conf pegasusrc \-\-isuuid 316f2986\-7754\-44ec\-8b38\-fcd0cb602ce0 .fi .if n \{\ .RE .\} .sp Runs pegasus\-statistics over a workflow run identified by a multiple workflow UUID: .sp .if n \{\ .RS 4 .\} .nf $ pegasus\-statistics \-\-conf pegasusrc \-\-isuuid 316f2986\-7754\-44ec\-8b38\-fcd0cb602ce0 \e 7ef77af8\-4eb2\-45ca\-b37d\-c5a02186133a .fi .if n \{\ .RE .\} .sp Runs pegasus\-statistics over all workflows in the STAMPEDE database: .sp .if n \{\ .RS 4 .\} .nf $ pegasus\-statistics \-\-conf pegasusrc \-\-multiple\-wf .fi .if n \{\ .RE .\} .SH "AUTHORS" .sp Prasanth Thomas Rajiv Mayani .sp Pegasus Team \m[blue]\fBhttp://pegasus\&.isi\&.edu\fR\m[]