Scroll to navigation

CFDUMP(1) cfdump CFDUMP(1)

NAME

cfdump - view CF fields

SYNOPSIS

cfdump [-1] [-c] [-h] [-i] [-l] [-n] [-s] [-u] [-x] [OPTIONS] INPUTS

DESCRIPTION

The cfdump tool generates text representations on standard output of the CF fields contained in the INPUTS (which may include directories if the --recursive option is set).

Accepts CF-netCDF and CFA-netCDF files (or URLs if DAP access is enabled), Met Office (UK) PP files and Met Office (UK) fields files as input. Multiple input files in a mixture of formats may be given and normal UNIX file globbing rules apply.

By default the contents of each input file is aggregated (i.e. combined) into as few multi-dimensional CF fields as possible. Unaggregatable fields in the input files may be omitted from the output (see the -x option). Information on which fields are unaggregatable, and why, may be displayed (see the --info option). All aggregation may be turned off with the -n option, in which case all input fields are output without modification.

See the AGGREGATION section for details on the aggregation process and unaggregatable fields.

By default each input file is treated separately. In this case there is no inter-file aggregation, but the contents of each file is aggregated independently of the others.

Alternatively, all of the input files may be treated collectively as a single CF dataset (see the -1 option). In this case aggregation is attempted within and between the input files.

In the displayed output, each component of a field is assigned one of the following roles as defined by the CF data model:

Field
Axis
Cell method
Dimension coordinate
Auxiliary coordinate
Cell measure
Coordinate reference

A field and its components are identified, where appropriate, by their standard names if available, or their long names. If neither is present then a netCDF variable name is used. Long names and netCDF variable names are preceded by long_name: and ncvar: respectively. Axis identities are inferred from the coordinates which span them.

Each data array of a field and its components is described by its dimensionality, units and (depending on the extent of the output requested) its first and last values. For longer and complete outputs, arrays containing coordinate bounds are given in the same way and included as part of their coordinates' descriptions.

AGGREGATION

Aggregation of input fields into as few multi-dimensional CF fields as possible is carried out according to the aggregation rules documented in CF ticket #78 (http://kitt.llnl.gov/trac/ticket/78). For each input field, the aggregation process creates a structural signature which is essentially a subset of the metadata of the field, including coordinate metadata and other domain information, but which contains no data values. The structural signature accounts for the following standard CF properties:

add_offset, calendar, cell_methods, _FillValue, flag_masks, flag_meanings, flag_values, missing_value, scale_factor, standard_error_multiplier, standard_name, units, valid_max, valid_min, valid_range

Aggregation is then attempted on each group of fields with the same, well defined structural signature, and will succeed where the coordinate data values imply a safe combination into a single dataset.

Not all fields are aggregatable. Unaggregatable fields are those without a well defined structural signature; or those with the same structural signature when at least two of them 1) can't be unambiguously distinguished by coordinates or other domain information or 2) contain coordinate reference fields or ancillary variable fields which themselves can't be unambiguously aggregated.

EXAMPLES

In these examples, a complete dataset has been split across two files (file1.nc and file2.nc). These may be passed to cfdump which can return a description of the recombined, aggregated field. The -1 option is necessary here so that the two input files are treated as parts of the same dataset.

The -s option displays the short, one-line output, which gives the identity of the field (air_temperature), the identities and sizes of its data array dimensions (time, latitude and longitude with sizes 1200, 64 and 128 respectively) and the units of the field's data array (K):



   $ cfdump -1s file1.nc file2.nc


   <CF Field: air_temperature(time(1200), latitude(64), longitude(128)) K>

The default summary gives the same information as the the one-line output, along with short descriptions of the field's other components:



   $ cfdump -1 file[12].nc


   air_temperature field summary


   -----------------------------


   Data           : air_temperature(time(1200), latitude(64), longitude(128)) K


   Cell methods   : time: mean (interval: 1.0 month)


   Axes           : time(12) = [ 450-11-16 00:00:00, ...,  550-10-16 12:00:00] noleap calendar


                  : latitude(64) = [-87.8638000488, ..., 87.8638000488] degrees_north


                  : longitude(128) = [0.0, ..., 357.1875] degrees_east


                  : height(1) = [2.0] m

This shows that the aggregated field has a cell method and four dimension coordinates, one of which (height) is a coordinate for a size 1 dimension that is not a dimension of the field's data array. The units and first and last values of the coordinates' data arrays are given and relative time values are translated into strings.

The -l option displays the long output, which includes each component's properties, as well as the first and last values of the field's data array:



   $ cfdump -1l file[1-2].nc


   ======================


   Field: air_temperature


   ======================


   Axes


       height(1)


       latitude(64)


       longitude(128)


       time(12)


   


   Data(time(12), latitude(64), longitude(128)) = [[[236.512756348, ..., 256.93371582]]] K


   cell_methods = time: mean (interval: 1.0 month)


   


   experiment_id = 'pre-industrial control experiment'


   long_name = 'air_temperature'


   missing_value = 1e+20


   standard_name = 'air_temperature'


   


   Dimension coordinate: time


       Data(time(12)) = [ 450-11-16 00:00:00, ...,  550-10-16 12:00:00] noleap calendar


       Bounds(time(12), 2) = [[ 450-11-01 00:00:00, ...,  550-11-01 00:00:00]] noleap calendar


       axis = 'T'


       standard_name = 'time'


   


   Dimension coordinate: latitude


       Data(latitude(64)) = [-87.8638000488, ..., 87.8638000488] degrees_north


       Bounds(latitude(64), 2) = [[-90.0, ..., 90.0]] degrees_north


       axis = 'Y'


       standard_name = 'latitude'


   


   Dimension coordinate: longitude


       Data(longitude(128)) = [0.0, ..., 357.1875] degrees_east


       Bounds(longitude(128), 2) = [[-1.40625, ..., 358.59375]] degrees_east


       axis = 'X'


       standard_name = 'longitude'


   


   Dimension coordinate: height


       Data(height(1)) = [2.0] m


       axis = 'Z'


       positive = 'up'


       standard_name = 'height'

In this case, the complete output using the -c option would be the same as the long output, since there are no fields contained in coordinate references nor ancillary variables.

OPTIONS

-1, --one
Treat all input files collectively as a single CF dataset. In this case aggregation is attempted within and between the input files.
Aggregation configuration: Create a new axis for each input field which has given property. If an input field has the property then, prior to aggregation, a new axis is created with an auxiliary coordinate whose data array is the property's value. This allows for the possibility of aggregation along the new axis. The property itself is deleted from that field. No axis is created for input fields which do not have the specified property.

Multiple axes may be created by specifying more than one --axis option.

For example, if you wish to aggregate an ensemble of model experiments that are distinguished by the source property, you can use --axis=source to create an ensemble axis which has an auxiliary coordinate variable containing the source property values.

Display complete outputs. Fields are described without abbreviation with the exception of data arrays, which are abbreviated to their first and last values. Fields contained in transforms and ancillary variables are given as long outputs (see the -l option).
Aggregation configuration: Requires that aggregated fields have adjacent dimension coordinate cells which partially overlap or share common boundary values. Ignored if the dimension coordinates do not have bounds.
Aggregation configuration: Require that an input field may only be aggregated with other fields if they all have the given CF property (standard or non-standard) with equal values. Ignored for any input field which does not have this property, or if the property is already accounted for in the structural signature.

Supersedes the behaviour for the given property that may be implied by the --exist_all option.

Multiple properties may be set by specifying more than one --equal option.

Aggregation configuration: Require that an input field may only be aggregated with other fields that have the same set of CF properties (excluding those already accounted for in the structural signature) with equal sets of values.

The behaviour for individual properties may be overridden by the --exist --ignore options.

For example, to insist that a group of aggregated input fields must all have the same CF properties (other than those accounted for in the structural signature) with matching values, but allowing the long_name properties have unequal values, you can use --equal_all --exist=long_name

Aggregation configuration: Require that an input field may only be aggregated with other fields if they all have the given CF property (standard or non-standard), but not requiring the values to be the same. Ignored for any input field which does not have this property, or if the property is already accounted for in the structural signature.

Supersedes the behaviour for the given property that may be implied by the --equal_all option.

Multiple properties may be set by specifying more than one --exist option.

Aggregation configuration: Require that an input field may only be aggregated with other fields that have the same set of CF properties (excluding those already accounted for in the structural signature), but not requiring the values to be the same.

The behaviour for individual properties may be overridden by the --equal --ignore options.

For example, to insist that a group of aggregated input fields must all have the same CF properties (other than those accounted for in the structural signature), regardless of their values, but also insisting that the long_name properties have equal values, you can use --exist_all --equal=long_name

In combination with --recursive also search for files in directories which resolve to symbolic links. Files specified by the INPUTS which are symbolic links are always followed. Note that setting --recursive --follow_symlinks can lead to infinite recursion if a directory which resolves to a symbolic link points to a parent directory of itself.
Display this man page.
Aggregation configuration: In the absence of standard names, allow fields and their components (such as coordinates) to be identified by their long_name CF properties or else their netCDF file variable names.
Aggregation configuration: An input field may be aggregated with other fields regardless of whether or not they have the given CF property (standard or non-standard) and regardless of its values. Ignored for any input field which does not have this property, or if the property is already accounted for in the structural signature.

This is the default behaviour in the absence of all the --exist --equal --exist_all --equal_all options and supersedes the behaviour for the given property that may be implied if any of these options are set.

Multiple properties may be set by specifying more than one --ignore option.

For example, to insist that a group of aggregated input fields must all have the same CF properties (other than those accounted for in the structural signature) with the same values, but with no restrictions on the existence or values of the long_name property you can use --equal_all --ignore=long_name

Ignore, without failing, any file which causes an error whilst being read, as would be the case for an empty file, unknown file format, etc. By default, an error occurs and the return code is non-zero.
Print information about the aggregation process. If N is 0 then no information is displayed. If N is 1 or more then display information on which fields are unaggregatable, and why. If N is 2 or more then display the field structural signatures and, when there is more than one field with the same structural signature, their canonical first and last coordinate values. If N is 3 or more then display the field complete aggregation metadata.

By default N is 1.

Display long outputs. Differs from the complete output (see the -c option) only in that fields contained in transforms and ancillary variables are given as one-line summaries (see the -s option) rather than as long outputs.
Aggregation configuration: Force fields and their components (such as coordinates) to be identified by their netCDF file variable names.
Aggregation configuration: Do not aggregate fields. Displays the input fields as they exist in the input files.
Aggregation configuration: Requires that aggregated fields have adjacent dimension coordinate cells which do not overlap (but they may share common boundary values). Ignored if the dimension coordinates do not have bounds.
Promote field components to independent top-level fields. If component is ancillary then ancillary data fields are promoted. If component is auxiliary then auxiliary coordinate variables are promoted. If component is measure then cell meausure variables are promoted. If component is reference then fields pointed to from formula_terms attributes are promoted. If component is field then all component fields are promoted.

Multiple conponent types may be promoted by specifying more than one --promote option.

For example, promote ancillary data field and cell measure variables to independent, top-level fields you can use --promote=ancillary --promote=measure

Aggregation configuration: Take into account the CF properties valid_max, valid_min and valid_range during aggregation. By default they are ignored for the purposes of aggregation and deleted from any aggregated output CF fields.
Display short outputs. Fields are described by a short, one-line summaries.
Aggregation configuration: Match axes between a field and its contained ancillary variable and coordinate reference fields via their netCDF dimension names and not via their domains.
Remove size 1 axes from the displayed field data arrays. If a size one axis has any one dimensional coordinates then these are converted to CF scalar coordinates.
Allow directories to be specified by the INPUTS and recursively search the directories for actual files to read. Set the --ignore_read_error option to bypass any unreadable files and the --follow_symlinks option to allow directories to be symbolic links.
Aggregation configuration: Assume that fields or their components (such as coordinates) with the same standard name (or other identifiers, see the -i option) but missing units all have equivalent (but unspecified) units, so that aggregation may occur. This is the default for Met Office (UK) PP files and Met Office (UK) fields files, but not for other formats.
Include size 1 axes in the displayed field data arrays. If a size one axis has any CF scalar coordinates then these are converted to one dimensional coordinates.
For Met Office (UK) PP files and Met Office (UK) fields files only, the Unified Model (UM) version to be used when decoding the header. Valid versions are, for example, 4.2, 6.6.3 and 8.2. The default version is 4.5. In general, the given version is ignored if it can be inferred from the header (which is usually the case for files created by the UM at versions 5.3 and later). The exception to this is when the given version has a third element (such as the 3 in 6.6.3), in which case any version in the header is ignored. This option is ignored for input files which are not Met Office (UK) PP files or Met Office (UK) fields files.
Aggregation configuration: Omit unaggregatable fields from the output. Ignored if the -n option is set. See the AGGREGATION section for the definition of an unaggregatable field.

SEE ALSO

cfa(1), ncdump(1)

LIBRARY

cf-python library version 1.3.1

BUGS

Reports of bugs are welcome at http://cfpython.bitbucket.org

LICENSE

Open Source Initiative MIT License

AUTHOR

David Hassell

1.3.1 2016-09-09