NAME¶
Funview - Database View Support for Tables
SYNOPSIS¶
This document contains a summary of the options for utilizing database-inspired
Views of tables.
DESCRIPTION¶
Database Views
In database parlance, a
View defines a "virtual table", i.e., a
description of row and/or column selection filters (but with no permanent
storage space allocated). When used in place of a table, a View selects the
specified rows and/or columns from one or more real tables. Views enable you
to see complicated data tables in a more convenient format. They also can be
used as a security mechanism, by restricting user access to specific columns
and/or rows. [See:
http://www.cs.unibo.it/~ciaccia/COURSES/RESOURCES/SQLTutorial/sqlch5.htm
for a good discussion of SQL Views.]
Funtools supports an expanded notion of Views for all tabular data (FITS tables,
raw binary tables, and ASCII column files). Funtools Views allow you to
pre-set values for the filter specification, the columns to activate, and
display format (though the latter is for fundisp only). Setting the filter and
column activation values provides functionality equivalent to that of a
classical database View, while the ability to set the format is similar to
classical report writing capabilities.
Funtools View Attributes
A Funtools View is a text file containing one or more of the following columns:
column description
------ -----------------------------
view name of view
file data file name or template
filter filter specification
columns columns to activate
format fundisp format specification
All of the attribute columns are optional, including the
view name
itself. This means that a View can be named or unnamed. Unnamed Views can
refer to a specific file or a template of files (obviously if neither the view
or the file column is specified, the input View specification will never be
used). You can specify any combination of filter, column, and format
parameters. (It also is possible to apply file-specific View to other files;
see the discussion on
View Lists below). Each column has a size limit
of 1024 characters.
For example, consider the following View file:
view file format columns filter
---- ---------------------- ------ ------------ -------
x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1
x2 ${HOME}/data/snr.ev x y pi pha cir 512 512 .1
x1 ${HOME}/data/snr.ev cir 512 512 .1
x1a ${HOME}/data/snr.ev x y pi pha
x0 ${HOME}/data/snr.ev
xf I=%4d
xc x y pi pha
xr cir 512 512 .1
*.ev x y pi pha
*.fit x y dx dy cir 400 400 3
*.fits I=%3d x y dx dy cir 400 400 3
This database example is in rdb format, i.e. using tab delimiters and permitting
null values. Any valid ASCII table format is acceptable, but if you use a
format that does not permit null values, it will be necessary to quote the
null strings.
The first five entries (x3, x2, x1, x1a, x0) are named entries defining default
values specifically for the snr.ev data file. Typically, you would use these
Views by specifying View name, and the corresponding file, filter, column, and
format values would be used. Note that the x0 View is essentially an alias for
the pathname of this file.
The next three entries define defaults that can be applied to any file. You
typically would use these View names in conjunction with a specific file name
(see
View Lists below) so that the associated parameter(s) were applied
to that file.
The last three entry in the database define unnamed Views that pertains to all
files ending with the specified templates. In these cases, any View that
specifies a file name matching the file template would be processed with the
associated parameter attributes.
Invoking a Funtools View (in Place of an Input File)
To use a Funtools View, you simply pre-pend the "v:" prefix to a View
name or a file name where an input file name usually is specified. For
example:
fundisp v:x3
specifies that the View named x3 (with its file name and associated parameters)
is processed as the input file to fundisp. Using the example database, above,
this is equivalent to:
fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha"
That is, the format is used with fundisp's \-f (format) switch, while the
filename and extension are composed of the x3 View's filename and region
filter.
Similarly, executing a command such as:
fundisp v:foo.fit
will match the unnamed View associated with the template "*.fit". This
is equivalent to executing:
fundisp foo.fit'[cir 400 400 3]' "x y dx dy"
Of course, if you omit the "v:" prefix, then no View processing takes
place:
fundisp foo.fit # process foo.fit without any View parameters
fundisp x3 # error (assuming there is no file named x3)
Basic View Matching Rules
When a "v:" prefix is recognized, Funtools searches for a View
database file in the following order:
location description
------------ ------------------------------------
FUN_VIEWFILE environment variable (any file name)
./.funtools.vu hidden file, default name
$HOME/.funtools.vu hidden file, default name
The first View database file located is used to construct a new filename, as
well as an activation column specification and a format specification. The
following rules are used:
1. An attempt is made to match the input name (i.e., the part of the input View
after the "v:" prefix) against the
view column value (if
present) of each row in the database. If a match is found, the values of all
non-blank columns are saved for later use. Also note that the first match
terminates the search: i.e., the order of the database rows matters.
2. If no
view match is made, an attempt is made to match the input name
against the
file column value (if present). Matching is performed on
the full pathname of both the input name and the database file name, and on
the non-directory (root) part of these files. This means that the root
specification:
fundisp v:snr.ev
will match a row in the database that has a full pathname in the file, allowing
you to use a
file\-matched View without having to specify the full
pathname. In this example, the "v:snr.ev" View specification will
match the first row (v:x3) in the database:
x3 ${HOME}/data/snr.ev I=%4d x y pi pha cir 512 512 .1
even though the row contains a fully qualified pathname as the file value. Once
again, values of all non-blank columns are saved, and the first match
terminates the search.
3. If neither a
view or a
view match has been found, then a simple
template match is attempted against the
view values. Template matching
supports a simplified version of file globbing (not a regular expression),
with support for a single "*" (all characters), "?"
(single character), or "[...]" (range) specification.
4. If no template match was found on the
view column, then a simple
template match is attempted against the
file columns.
5. If no match is found, then the filename (minus the "v:" prefix) is
returned.
More on View Matching Rules - Single vs. Multiple Matches
The matching rules described above stop after the first match, regardless of
whether that match provides values for all three parameters (filter, columns,
and format). In cases where a
view or
file match does not
provide all three values, it is possible that a template match might do so.
With regard to the example View database above, the x1 View provides only a
filter, while omitting both the format and columns values. But note that the
final rows in the database could provide the values via a template match on
the filename. This sort of multiple matching is especially valuable in order
to provide "global" values to several Views.
Obviously, multiple matching might not be wanted in every case. Therefore, we
support both multiple matching and single matching according to the value of
the FUN_VIEWMATCH environment variable. If the FUN_VIEWMATCH environment
variable exists and if its value begins with "s", then a single
match is used and missing parameters are not filled in with subsequent
template matches on the file name. That is, matching rules above are followed
exactly as explained above. If the value of this environment variable begins
with "m" (or does not exist), then multiple matches are used to try
to fill in missing parameters. In this case, template matching always takes
place and missing values are taken from these template matches.
Thus, in the example above, the View specification:
fundisp v:x1
will take the file name and filter value from the x1 View:
x1 ${HOME}/data/snr.ev cir 512 512 .1
The column value then will be taken from the "*.ev" file template
match against the x1 file name:
*.ev x y pi pha
Note once again that order is important: missing values are taken in the order
in which the template matches are processed.
View Lists - Applying a View to Any File
It is possible to apply a named View, or even several Views, to any data file by
appending a
viewlist immediately after the standard "v:"
prefix. A viewlist takes the form:
:v1,v2,...vn:
where v1, v2, etc. are named Views. The two ":" colon characters
surrounding the list are required. Thus, the syntax for applying a viewlist to
a file is:
v::view1,view2,...viewn:filename
Note that the name after the last ":" is assumed to be a file; it is
not permissible (or sensible) to use a View name.
For example, the View specification:
fundisp v::x2:foo
applies the x2 View to the file foo (even if there is a View named foo) and (in
using our example database) is equivalent to:
./fundisp foo'[cir 512 512 .1] "x y pi pha"
The same command can be effected using a list of Views:
fundisp v::x1,x1a:foo
What happens if a viewlist is used and the file also matches a template?
Consider, for example, this View specification:
fundisp v::x2:foo.fit
Here, the x2 View will supply filter and column values, while the template *.fit
can also supply (different) filter and column values. In this case, the
explicitly specified Views of the viewlist trump the matched view values.
On the other hand, if a file template match can supply a View value that is not
supplied by the viewlist, then that value will be taken from the file template
match. For example:
fundisp v::x2:foo.fits
does not explicitly supply a format value, but the file match on *.fits can and
does. You can avoid supplying missing values using file template matching by
replacing the first ":" with a "-" in a viewlist
specification:
fundisp v:-x2:foo.fits
The use of ":+" to explicitly allow file template matching is also
supported, but is the same as the default case. Note that the nuances of
viewlist support are subject to change as our experience and understanding
grow.
Overriding Values Associated with a View
To override values associated with a View, simply supply the override values in
the correct place on the command line. Thus, given the example database
described above, the command:
fundisp v:x3
specifies that the View named x3, along with its file name and associated
parameters, be processed as the input file to fundisp in this way:
fundisp -f "I=%4d" ${HOME}/data/snr.ev'[cir 512 512 .1]' "x y pi pha"
To override one or more of these values, simply specify a new value for the
format, filter, or columns. For example, if your input View file contains a
filter, then the View will use that filter as an override of the View filter:
fundisp v:x3'[cir 400 400 3]'
will use the columns and format of the x3 View but not the x3 filter. Further
examples are:
fundisp v:x3 "x y dx dy" # activate a different set of columns
fundisp -f "I=%3d" v:x3 # use a different format statement
Note that extension names, extension index values, and other non-filter
specifications
do not override the View filter. Thus:
fundisp v:foo.fit[3]
will still use the filter associated with the .fit template (see above), since
the "3" is an extension index, not a filter.
Environment Variables
The following environment variables are used by Funtools Views:
- •
- FUN_VIEWNAME
The FUN_VIEWNAME environment variable specifies the name and location
of the View database file. If not present, the files ./.funtools.vu and
$HOME/.funtools.vu are searched for, in that order.
- •
- FUN_VIEWMATCH
The FUN_VIEWMATCH environment variable specifies whether a single
match or multiple match algorithm is used to locate parameter values. If
the value of this environment variable begins with "s", then a
single match is used and missing parameters are not filled in with
subsequent template matches on the file name. If the value begins with
"m", then multiple matches are used to try to fill in missing
parameters. The default is to use multiple matches.
Restrictions and Problems
Support for overriding a filter (while not overriding extension names, extension
indexes, etc.) requires that we can sense the presence of a filter in a
bracket specification. It is unclear yet whether our algorithm is perfect.
Go to Funtools Help Index
Last updated: August 3, 2007