.TH X2SYS_DATALIST 1gmt "Feb 27 2014" "GMT 4.5.13 (SVN)" "Generic Mapping Tools"
.SH NAME
x2sys_datalist \- A generic data-extractor for ASCII or binary files
.SH SYNOPSIS
\fBx2sys_datalist\fP \fItrack(s)\fP \fB\-T\fP\fITAG\fP [ \fB\-A\fP ] [ \fB\-F\fP\fIname1,name2,...\fP ] 
[ \fB\-H\fP[\fBi\fP][\fInrec\fP] ] [ \fB\-L\fP[\fIcorrtable\fP] ] [ \fB\-R\fP\fIwest\fP/\fIeast\fP/\fIsouth\fP/\fInorth\fP[\fBr\fP] ]  [ \fB\-S\fP ] [ \fB\-V\fP ] [ \fB\-bo\fP[\fBs\fP|\fBS\fP|\fBd\fP|\fBD\fP[\fIncol\fP]|\fBc\fP[\fIvar1\fP\fB/\fP\fI...\fP]] ] [ \fB\-m\fP[\fIflag\fP] ]
.SH DESCRIPTION
\fBx2sys_datalist\fP reads one or more files and produces a single ASCII [or binary] table. 
The files can be of any format, which must be described and passed with the
\fB\-T\fP option.  You may limit the output to a geographic region, and insist
that the output from several files be separated by a multiple segment header.
Only the named data fields will be output [Default selects all columns].
.TP
\fItracks\fP
Can be one or more ASCII, native binary, or COARDS netCDF 1-D data files.
To supply the data files via a text file with a list of tracks (one per record),
specify the name of the track list after a leading equal-sign
(e.g., =tracks.lis).  If the names are missing their file extension we will
append the suffix specified for this \fITAG\fP.  Track files will be searched
for first in the current directory and second in all directories listed in
\fB$X2SYS_HOME\fP/\fITAG\fP/\fITAG\fP_paths.txt (if it exists). [If \fB$X2SYS_HOME\fP
is not set it will default to \fB$GMT_SHAREDIR\fP/x2sys]. (Note: MGD77 files will
also be looked for via \fBMGD77_HOME\fP/mgd77_paths.txt and *.gmt files will be
searched for via \fB$GMT_SHAREDIR\fP/mgg/gmtfile_paths).
.TP
\fB\-T\fP
Specify the x2sys \fITAG\fP which tracks the attributes of this data type.
.SH OPTIONS
No space between the option flag and the associated arguments.
.TP
\fB\-A\fP
Eliminate COEs by distributing the COE between the two tracks in proportion to track weight.
These (dist, adjustment) spline knots files for each track and data column
are called \fItrack.column\fP.adj and are expected to be in the \fB$X2SYS_HOME\fP/\fITAG\fP directory.
The adjustments are only applied if the corresponding adjust file can be found [No residual adjustments]
.TP
\fB\-F\fP
Give a comma-separated sub-set list of column names defined in the definition
file. [Default selects all data columns].
.TP
\fB\-H\fP
Input file(s) has header record(s).  If used, the default number of header records is \fBN_HEADER_RECS\fP.
Use \fB\-Hi\fP if only input data should have header records [Default will write out header records if the
input data have them]. Blank lines and lines starting with # are always skipped.
.TP
\fB\-L\fP
Apply optimal corrections to columns where such corrections are available.  Append the correction
table to use [Default uses the correction table \fITAG\fP_corrections.txt which is expected to reside
in the \fB$X2SYS_HOME\fP/\fITAG\fP directory].
For the format of this file, see CORRECTIONS below.
.TP
\fB\-R\fP
\fIwest, east, south,\fP and \fInorth\fP specify the Region of interest, and you may specify them
in decimal degrees or in [+-]dd:mm[:ss.xxx][W|E|S|N] format.  Append \fBr\fP if lower left and upper right
map coordinates are given instead of w/e/s/n.  The two shorthands \fB\-Rg\fP and \fB\-Rd\fP stand for global domain
(0/360 and -180/+180 in longitude respectively, with -90/+90 in latitude). Alternatively, specify the name
of an existing grid file and the \fB\-R\fP settings (and grid spacing, if applicable) are copied from the grid.
For Cartesian data just give \fIxmin/xmax/ymin/ymax\fP.  This option limits
the COEs to those that fall inside the specified domain.
.TP
\fB\-S\fP
Suppress output records where all the data columns are NaN [Default will output all records].
.TP
\fB\-V\fP
Selects verbose mode, which will send progress reports to stderr [Default runs "silently"].
.TP
\fB\-bo\fP
Selects binary output.
Append \fBs\fP for single precision [Default is \fBd\fP (double)].
Uppercase \fBS\fP or \fBD\fP will force byte-swapping.
Optionally, append \fIncol\fP, the number of desired columns in your binary output file.
.TP
\fB\-m\fP
Output a multisegment header between data from each track.  Note this option does not
imply anything about the input file; that information is conveyed via the system tag (\fB\-T\fP).
.SH EXAMPLES
To extract all data from the old-style MGG supplement file c2104.gmt, recognized by the tag GMT:
.br
.sp
\fBx2sys_datalist\fP c2104.gmt \fB\-T\fP GMT > myfile
.br
.sp
To make lon,lat, and depth input for \fBblockmean\fP and \fBsurface\fP using all the files
listed in the file tracks.lis and define by the tag TRK, but only the data that are inside the specified
area, and make output binary, run
.br
.sp
\fBx2sys_datalist\fP =tracks.lis \fB\-T\fP TRK \fB\-F\fP lon,lat,depth \fB\-R\fP-40/-30/25/35 \fB\-bo\fP > alltopo_bin.xyz
.SH CORRECTIONS
The correction table is an ASCII file with coefficients and parameters needed to carry out corrections.
This table is usually produced by \fBx2sys_solve\fP.
Comment records beginning with # are allowed.  All correction records are of the form
.br
.sp
\fItrackID observation correction\fP
.br
.sp
where \fItrackID\fP is the track name, \fIobservation\fP is one of the abbreviations for an observed field
contained in files under this TAG, and \fIcorrection\fP consists of one or more white-space-separated \fIterm\fPs that will be
\fBsubtracted\fP from the observation before output.  Each \fIterm\fP must have this exact syntax:
.br
.sp
\fIfactor\fP[*[\fIfunction\fP]([\fIscale\fP](\fIabbrev\fP[-\fIorigin\fP]))[^\fIpower\fP]]
.br
.sp
where terms in brackets are optional (the brackets themselves are not used but regular parentheses must
be used exactly as indicated).  No spaces are allowed except between \fIterm\fPs. The \fIfactor\fP is the amplitude
of the basis function, while the optional \fIfunction\fP can be one of sin, cos, or exp. The
optional \fIscale\fP and \fIorigin\fP can be used to translate the argument (before giving it to the optional
function).  The argument \fIabbrev\fP is one of the abbreviations for columns known to this TAG.
However, it can also be one of the three auxiliary terms \fBdist\fP (for along-track distances), \fBazim\fP for
along-track azimuths, and \fBvel\fP (for along-track speed); these are all sensitive to the \fB\-C\fP and \fB\-N\fP
settings used when defining the TAB; furthermore, \fBvel\fP requires \fBtime\fP to be present in the data.  If \fIorigin\fP is
given as \fBT\fP it means that we should replace it with the value of \fIabbrev\fP for the very
first record in the file (this is usually only done for \fItime\fP).  If the first data record entry is
NaN we revert \fIorigin\fP to zero.  Optionally, raise the entire expression to the given \fIpower\fP,
before multiplying by \fIfactor\fP.  The following is an example of fictitious corrections to the
track ABC, implying the \fBz\fP column should have a linear
trend removed, the field \fBobs\fP should be corrected by a strange dependency on latitude,
\fBweight\fP needs to have 1 added (hence correction is given as -1), and \fBfuel\fP should be
reduced by a linear distance term:
.br
.sp
ABC z	7.1	1e-4*((time-T))
.br
ABC obs	0.5*exp(-1e-3(lat))^1.5
.br
ABC weight	-1
.br
ABC fuel 0.02*((dist))
.SH "SEE ALSO"
.IR blockmean (1),
.IR GMT (1),
.IR surface (1),
.IR x2sys_init (1),
.IR x2sys_datalist (1),
.IR x2sys_get (1),
.IR x2sys_list (1),
.IR x2sys_put (1),
.IR x2sys_report (1),
.IR x2sys_solve (1)