NAME¶
catmandu - LibreCat command line tools
SYNOPSIS¶
catmandu [-D|-D2|-D3] [-I path] [-L path] [COMMAND] [-?hqv] [long options...] [ARGS]
catmandu --version
catmandu help
# convert
catmandu convert JSON to YAML < file.json
catmandu convert YAML to JSON < file.yaml
catmandu convert YAML < file.yaml # 'to JSON' is implicit
catmandu convert MARC < records.mrc
catmandu convert MARC --fix 'marc_map(245,title)' < records.mrc
catmandu convert OAI --url https://biblio.ugent.be/oai
catmandu convert SRU --base http://www.unicat.be/sru --query dna
# import
catmandu import JSON to MongoDB --database_name mydb < records.json
catmandu import MARC to MongoDB --database_name mydb < records.mrc
catmandu import MARC to ElasticSearch --index_name mydb < records.mrc
catmandu import MARC to ElasticSearch --index_name mydb --fix 'marc_map("245a","title")' < records.mrc
# export
catmandu export MongoDB --database_name mydb --bag data to JSON
catmandu export MongoDB --database_name mydb --bag data to JSON --fix 'retain_field("_id")'
catmandu export Solr --url http://localhost:8983/solr to JSON
catmandu export ElasticSearch --index_name mydb to JSON
# configuration
$ cat catmandu.yml
---
store:
test1:
package: MongoDB
options:
database_name: mydb
catmandu import JSON to test1 < records.json
catmandu config # show the contents of catmandu.yml
catmandu count test1
catmandu delete test1 -q 'title:"My Rabbit"'
COMMANDS¶
help COMMAND¶
Documentation about command line options.
config¶
Display the contents of the catmandu.yml files in your project.
convert IMPORTER [OPTIONS] [[--fix FIX]] to EXPORTER [OPTIONS]¶
Convert data from one format to another format with help of Catmandu::Importers
and Catmandu::Exporters. All options given on the command line will be send to
the Importer or Exporter as initialization parameters. E.g.
catmandu convert OAI --url BASEURL --metadataPrefx PREFIX \
to \
CSV --header 1 --sep_char '|'
Look at the documentation of all the importers and exporters to see which
options are available.
All importers accept one or more --fix options with the name of a fix file or
file command.
catmandu convert JSON --fix fixes.txt
catmandu convert JSON --fix 'remove_field(abc)'
The JSON exporter is implicit and is optional.
catmandu convert YAML to JSON
catmandu convert YAML # shorter
count STORE [OPTIONS]¶
Count the number of items in a Catmandu::Store. All options given on the command
line will be send to the store as initialization parameters. E.g.
catmandu count MongoDB --database_name mydb --bag data
delete STORE [OPTIONS] [-q QUERY]¶
Delete items from a Catmandu::Store. All options given on the command line will
be send to the store as initialization parameters. Optionally a QUERY can be
provided to select the items to delete. E.g.
catmandu delete MongoDB --database_name mydb --bag data -q "title:Delete me"
export STORE [OPTIONS] [-q QUERY] [--limit LIMIT] [[--fix FIX]] to EXPORTER [OPTIONS]¶
Export items from a Catmandu::Store using a Catmandu::Exporter. All options
given on the command line will be send to the Store or Exporter as
initialization parameters. Optionally a QUERY and LIMIT can be provided to
selectively export items from the Store. Optionally a FIX can be provided to
be executed on all the items before they are exporter by the exporter.
E.g.
catmandu export MongoDB --database_name mydb --bag data -q "title:Perl"
import IMPORTER [OPTIONS] [[--fix FIX]] to STORE [OPTIONS]¶
Import items from a Catmandu::Importer to a Catmandu::Store. All options given
on the command line will be send to the Importer or Store as initialization
parameters. Optionally a FIX can be provided to be executed on all the items
before they are stored into the Store.
catmandu import JSON to MongoDB --database_name mydb < records.json
move STORE [OPTIONS] [-q QUERY] [--limit LIMIT] [[--fix FIX]] to STORE [OPTIONS]¶
Move items from one Catmandu::Store to another. All options given on the command
line will be send to the Stores as initialization parameters. Optionally a
QUERY and LIMIT can be provided to selectively export items from the Store.
Optionally a FIX can be provided to be executed on all the items before they
are stored.
data [LONG_OPTIONS]¶
Low level data manipulation command. See DATA OPTIONS below for full
documentation.
OPTIONS¶
- -v
- Verbose output.
- -h
- -?
- --lib-path PATH
- -I
- Specifies Perl library include paths, like perl's -I option. This option
is repeatable.
- --load-path PATH
- -L PATH
- Search upwards from this path for configuration files and accompanying lib
directories. This option is repeatable.
- --debug [LEVEL]
- -D[LEVEL]
- Set the debugging level for Catmandu. LEVEL is one of:
1 - WARN messages
2 - INFO messages
3 - DEBUG messages
Requires an installation of Log::Log4perl and Log::Any::Adapter::Log4perl.
By default an internal log4perl configuration file is used that sends all
debug messages to stderr. Optionally in the catmandu.yml the path to a
log4perl configuration file can be set. E.g.
catmandu.yml:
log4perl: /etc/log4perl.conf
or
log4perl: "
log4perl.category.Catmandu=$level,$appender
log4perl.appender.STDOUT=Log::Log4perl::Appender::Screen
log4perl.appender.STDOUT.stderr=0
log4perl.appender.STDOUT.utf8=1
log4perl.appender.STDOUT.layout=PatternLayout
log4perl.appender.STDOUT.layout.ConversionPattern=%d [%P] - %p %l %M time=%r : %m%n
log4perl.appender.STDERR=Log::Log4perl::Appender::Screen
log4perl.appender.STDERR.stderr=0
log4perl.appender.STDERR.utf8=1
log4perl.appender.STDERR.layout=PatternLayout
log4perl.appender.STDERR.layout.ConversionPattern=%d [%P] - %l : %m%n
"
See also Catmandu::Logger to enable logging for your own Catmandu
packages.
DATA OPTIONS¶
These low level options are available to the "catmandu data" command.
- --from-store STORE
- Read items from a Catmandu::Store such as: DBI, MongoDB, Solr,
ElasticSearch, ...
- --from-importer IMPORTER
- Read items from a Catmandu::Importer such as: YAML, JSON, CSV, MARC, OAI
...
- --from-bag BAG_NAME
- This option should be used in combination with a --from-store. Select from
the Store only the items belonging to the bag BAG_NAME.
- --from-OPTION OPTION_VALUE
- To include configuration options to a Catmandu::Store or
Catmandu::Importer an argument with prefix '--from-' can be used. E.g. to
provide an 'file' to an Importer use:
--from-file /my/path/to/file.txt
- --into-exporter EXPORTER
- Add each item read into a Catmandu::Exporter such as: JSON, YAML,
Template, XLS, ...
- --into-store STORE
- Add each item read into a Catmandu::Store such as: DBI, MongoDB, Solr,
ElasticSearch, ...
- --into-bag BAG_NAME
- This option should be used in combination with a --into-store. Add each
item into the Store with bag BAG_NAME.
- --into-OPTION OPTION_VALUE
- To include configuration options to a Catmandu::Store or
Catmandu::Exporter an argument with prefix '--into-' can be used. E.g. to
provide an 'file' to an Exporter use:
--into-file /tmp/file.txt
- --fix EXPRESSION
- --fix FILE
- When exporting or storing apply a fix EXPRESSION or all the fixes in FILE
on each item. This option is repeatable. FILE can also be an arbitrary
executable if Catmandu::Fix::cmd is installed.
- --start NUM
- When exporting or storing data skip the first NUM items.
- --total NUM
- When exporting of storing data process only a total NUM of items.
- --replace
- When storing data first delete all the items in the Store or Bag.
- --query QUERY
- --cql-query QUERY | -q QUERY
- When a Store is searchable execute and return the results. Using the
--cql-query or -q option the CQL query language can be used (if
supported).
- --limit NUM
- Limit the number of results returned from the searchable results to
NUM.
- --count
- Report the number of items processed.
- --version
- Return the installed version of Catmandu.
SEE ALSO¶
Catmandu::Importer Catmandu::Exporter Catmandu::Store Catmandu::MARC
Catmandu::OAI Catmandu::MongoDB Catmandu::Solr Catmandu::ElasticSearch