.\" Automatically generated by Pod::Man 2.28 (Pod::Simple 3.29)
.\"
.\" Standard preamble:
.\" ========================================================================
.de Sp \" Vertical space (when we can't use .PP)
.if t .sp .5v
.if n .sp
..
.de Vb \" Begin verbatim text
.ft CW
.nf
.ne \\$1
..
.de Ve \" End verbatim text
.ft R
.fi
..
.\" Set up some character translations and predefined strings.  \*(-- will
.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left
.\" double quote, and \*(R" will give a right double quote.  \*(C+ will
.\" give a nicer C++.  Capital omega is used to do unbreakable dashes and
.\" therefore won't be available.  \*(C` and \*(C' expand to `' in nroff,
.\" nothing in troff, for use with C<>.
.tr \(*W-
.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'
.ie n \{\
.    ds -- \(*W-
.    ds PI pi
.    if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch
.    if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\"  diablo 12 pitch
.    ds L" ""
.    ds R" ""
.    ds C` ""
.    ds C' ""
'br\}
.el\{\
.    ds -- \|\(em\|
.    ds PI \(*p
.    ds L" ``
.    ds R" ''
.    ds C`
.    ds C'
'br\}
.\"
.\" Escape single quotes in literal strings from groff's Unicode transform.
.ie \n(.g .ds Aq \(aq
.el       .ds Aq '
.\"
.\" If the F register is turned on, we'll generate index entries on stderr for
.\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index
.\" entries marked with X<> in POD.  Of course, you'll have to process the
.\" output yourself in some meaningful fashion.
.\"
.\" Avoid warning from groff about undefined register 'F'.
.de IX
..
.nr rF 0
.if \n(.g .if rF .nr rF 1
.if (\n(rF:(\n(.g==0)) \{
.    if \nF \{
.        de IX
.        tm Index:\\$1\t\\n%\t"\\$2"
..
.        if !\nF==2 \{
.            nr % 0
.            nr F 2
.        \}
.    \}
.\}
.rr rF
.\" ========================================================================
.\"
.IX Title "STAG-STORENODE 1p"
.TH STAG-STORENODE 1p "2016-05-29" "perl v5.22.2" "User Contributed Perl Documentation"
.\" For nroff, turn off justification.  Always turn off hyphenation; it makes
.\" way too many mistakes in technical documents.
.if n .ad l
.nh
.SH "NAME"
stag\-storenode.pl \- script is for storing data in database
.SH "SYNOPSIS"
.IX Header "SYNOPSIS"
.Vb 1
\&  stag\-storenode.pl \-d "dbi:Pg:dbname=mydb;host=localhost" myfile.xml
.Ve
.SH "DESCRIPTION"
.IX Header "DESCRIPTION"
This script is for storing data (specified in a nested file format
such as \s-1XML\s0 or S\-Expressions) in a database. It assumes a database
schema corresponding to the tags in the input data already exists.
.SS "\s-1ARGUMENTS\s0"
.IX Subsection "ARGUMENTS"
\fI\-d \f(BI\s-1DBNAME\s0\fI\fR
.IX Subsection "-d DBNAME"
.PP
This is either a \s-1DBI\s0 locator or the logical name of a database in the
\&\s-1DBSTAG_DBIMAP_FILE\s0 config file
.PP
\fI\-user \f(BI\s-1USER\s0\fI\fR
.IX Subsection "-user USER"
.PP
db user name
.PP
\fI\-password \f(BI\s-1PASSWORD\s0\fI\fR
.IX Subsection "-password PASSWORD"
.PP
db user password
.PP
\fI\-u \f(BI\s-1UNIT\s0\fI\fR
.IX Subsection "-u UNIT"
.PP
This is the node/element name on which to load; a database loading
event will be fired every time one of these elements is parsed; this
also constitutes a whole transaction
.PP
\fI\-c \f(BI\s-1STAGMAPFILE\s0\fI\fR
.IX Subsection "-c STAGMAPFILE"
.PP
This is a stag mapping file, indicating which elements are aliases
.PP
\fI\-p \f(BI\s-1PARSER\s0\fI\fR
.IX Subsection "-p PARSER"
.PP
Default is xml; can be any stag compatible parser, \s-1OR\s0 a perl module
which will parse the input file and fire stag events (see
Data::Stag::BaseGenerator)
.PP
\fI\-t \f(BI\s-1TRANSFORMER\s0\fI\fR
.IX Subsection "-t TRANSFORMER"
.PP
This is the name of a perl module that will perform a transformation
on the stag events/XML. See also stag\-handle.pl
.PP
\fI\-noupdate \f(BI\s-1NODELIST\s0\fI\fR
.IX Subsection "-noupdate NODELIST"
.PP
A comma-seperated (no spaces) list of nodes/elements on which no
update should be performed if a unique key is found to be present in
the \s-1DB\s0
.PP
\fI\-trust_ids\fR
.IX Subsection "-trust_ids"
.PP
If this flag is present, the values for primary key values are
trusted; otherwise they are assumed to be surrogate internal IDs that
should not be used. In this case they will be remapped.
.PP
\fI\-tracenode \f(BI\s-1TABLE/COLUMN\s0\fI\fR
.IX Subsection "-tracenode TABLE/COLUMN"
.PP
E.g.
.PP
.Vb 1
\&  \-tracenode person/name
.Ve
.PP
Writes out a line on \s-1STDERR\s0 for every new person inserted/updated
.PP
\fI\-cache \f(BI\s-1TABLE\s0\fI=\f(BI\s-1MODE\s0\fI\fR
.IX Subsection "-cache TABLE=MODE"
.PP
Can be specified multiple times
.PP
Example:
.PP
.Vb 1
\&  \-cache 
\&
\&                   0: off (default)
\&                   1: memory\-caching ON
\&                   2: memory\-caching OFF, bulkload ON
\&                   3: memory\-caching ON, bulkload ON
.Ve
.PP
IN-MEMORY \s-1CACHING\s0
.PP
By default no in-memory caching is used. If this is set to 1,
then an in-memory cache is used for any particular element. No cache
management is used, so you should be sure not to cache elements that
will cause memory overloads.
.PP
Setting this will not affect the final result, it is purely an
efficiency measure for use with \fIstorenode()\fR.
.PP
The cache is indexed by all unique keys for that particular
element/table, wherever those unique keys are set
.PP
\&\s-1BULKLOAD\s0
.PP
If bulkload is used without memory-caching (set to 2), then only
INSERTs will be performed for this element. Note that this could
potentially cause a unique key violation, if the same element is
present twice
.PP
If bulkload is used with memory-caching (set to 3) then only INSERTs
will be performed; the unique serial/autoincrement identifiers for
those inserts will be cached and used. This means you can have the
same element twice. However, the load must take place in one session,
otherwise the contents of memory will be lost
.SH "XML TO DB MAPPING"
.IX Header "XML TO DB MAPPING"
See DBIx::DBStag for details of the actual mapping. Two styles of
mapping are allowed: stag-dbxml and XORT-style \s-1XML.\s0 You do not have to
specify which, they are sufficiently similar that the loader can
accept either.
.SH "MAKING DATABASE FROM XML FILES"
.IX Header "MAKING DATABASE FROM XML FILES"
It is possible to automatically generate a database schema and
populate it directly from \s-1XML\s0 files (or from Stag objects or other
Stag compatible files). Of course, this is no substitute for proper
relational design, but often it can be necessary to quickly generate
databases from heterogeneous \s-1XML\s0 data sources, for the purposes of
data mining.
.PP
There are 3 steps involved:
.PP
1. Prepare the input \s-1XML \s0(for instance, modifying db reserved words).
2. Autogenerate the \s-1CREATE TABLE\s0 statements, and make a db from these.
3. Store the \s-1XML\s0 data in the database.
.SS "Step 1: Prepare input file"
.IX Subsection "Step 1: Prepare input file"
You may need to make modifications to your \s-1XML\s0 before it can be used
to make a schema. If your \s-1XML\s0 elements contain any words that are
reserved by your \s-1DB\s0 you should change these.
.PP
Any \s-1XML\s0 processing tool (eg \s-1XSLT\s0) can be used. Alternatively you can
use the script 'stag\-mogrify'
.PP
e.g. to get rid of '\-' characters (this is how Stag treates
attributes) and to change the element with postgresql reserved word
\&'date', do this:
.PP
.Vb 1
\&  stag\-mogrify.pl \-xml \-r \*(Aqs/^date$/moddate/\*(Aq \-r \*(Aqs/\e\-//g\*(Aq data.xml > data.mog.xml
.Ve
.PP
You may also need to explicitly make elements where you will need
linking tables. For instance, if the relationship between 'movie' and
\&'star' is many-to-many, and your input data looks like this:
.PP
.Vb 4
\&  (movie
\&   (name "star wars")
\&   (star
\&    (name "mark hamill")))
.Ve
.PP
You will need to *interpose* an element between these two, like this:
.PP
.Vb 5
\&  (movie
\&   (name "star wars")
\&   (movie2star
\&    (star
\&     (name "mark hamill"))))
.Ve
.PP
you can do this with the \-i switch:
.PP
.Vb 1
\&  stag\-mogrify.pl \-xml \-i movie,star,movie2star data.xml > data.mog.xml
.Ve
.PP
or if you simply do:
.PP
.Vb 1
\&  stag\-mogrify.pl \-xml \-i star data.xml > data.mog.xml
.Ve
.PP
the mogrifier will simply interpose an element above every time it
sees 'star'; the naming rule is to use the two elements with an
underscore between (in this case, 'movie_star').
.SS "Step 2: Generating \s-1CREATE TABLE\s0 statements"
.IX Subsection "Step 2: Generating CREATE TABLE statements"
Use the stag\-autoddl.pl script;
.PP
.Vb 1
\&  stag\-autoddl.pl data.mog.xml > table.sql
.Ve
.PP
The default rule is to create foreign keys from the nested element to
the outer element; you will want linking tables tobe treated
differently (a linking table will point to parent and child elements).
.PP
.Vb 1
\&  stag\-autoddl.pl \-l movie2star \-l star2character data.mog.xml > table.sql
.Ve
.PP
Once you have done this, load the statements into your db; eg for postgresql
(for other databases, use SQL::Translator)
.PP
.Vb 1
\&  psql \-a mydb < table.sql
.Ve
.PP
If something goes wrong, go back to step 1 and sort it out!
.PP
Note that certain rules are followed: ever table generated gets a
surrogate primary key of type 'serial'; this is used to generate
foreign key relationships. The rule used is primary and foreign key
names are the name of the table with the '_id' suffix.
.PP
Feel free to modify the autogenerated schema at this stage (eg add
uniqueness constraints)
.SS "Step 3: Store the data in the db"
.IX Subsection "Step 3: Store the data in the db"
.Vb 1
\&  stag\-storenode.pl \-u movie \-d \*(Aqdbi:Pg:mydb\*(Aq data.mog.xml
.Ve
.PP
You generally don't need extra metadata here; everything can be
infered by introspecting the database.
.PP
The \-u|unit switch controls when transactions are committed
.PP
You can omit the \-u switch, and every node directly under the top node
will be stored. This will also be the transaction unit.
.PP
If this works, you should now be able to retrieve \s-1XML\s0 from the database, eg
.PP
.Vb 1
\&  stag-selectall_xml \-d \*(Aqdbi:Pg:mydb\*(Aq \*(AqSELECT * FROM x NATURAL JOIN y\*(Aq
.Ve