table of contents
- NAME
- SYNOPSIS
- DESCRIPTION
- The [common] block
- PBS options
- Condor options
- SGE options
- SLURM options
- LSF options
- LL options
- Desktop Grid options
- Boinc options
- Other [common] options
- [vo] block
- [group] Authorisation block
- The [grid-manager] block
- [data-staging] block
- [gridftpd] block
- [gridftpd/filedir] block
- [gridftpd/jobs] subblock
- [infosys] block
- BDII specific
- EGIIS-related commands
- [infosys/glue12] block
- [infosys/site/sitename] block
- [infosys/admindomain] block
- [infosys/index/indexname] block
- [infosys/index/indexname/registration/registrationname] block
- [cluster] block
- [infosys/cluster/registration/registrationname] block
- [queue/queue_name] block
- [registration/emir] block
- [nordugridmap] block
- [acix/cacheserver] block
- [acix/indexserver] block
- [gangliarc] block
arc.conf(5) | NorduGrid ARC | arc.conf(5) |
NAME¶
arc.conf - ARC configurationSYNOPSIS¶
/etc/arc.conf${ARC_LOCATION}/etc/arc.conf
DESCRIPTION¶
ARC has two separate configuration files - one for client tools and another for services. This document describes the services configuration file. For client configuration please see "ARC Clients User Manual" at http://www.nordugrid.org/documents/arc-ui.pdfARC configuration uses a plain-text "ini-style" format. It is also possible to use an XML format, however that is outside the scope of this document.
The configuration file consists of several configuration blocks. Each configuration block is identified by a keyword and contains the configuration options for a specific part of the ARC middleware.
Each configuration block starts with its identifying keyword inside square brackets. Thereafter follows one or more attribute value pairs written one on each line in the following format (note that the attribute names are CASE-SENSITIVE):
[keyword1] attribute1="value1" attribute2="value2" [keyword2] attribute="value"
If the ARC_LOCATION environment variable is set the ARC configuration file located at ${ARC_LOCATION}/etc/arc.conf is read first. If this file is not present or the relevant configuration information is not found in this file, the file at /etc/arc.conf is read.
The [common] block¶
The parameters set within this block are available for all the other blocks. These are the configuration parameters shared by the different components of ARC (e.g. grid-manager, infosys)- hostname
- hostname - the FQDN of the frontend node, optional in the common block but
MUST be set in the cluster block
Example:
hostname="myhost.org" - x509_voms_dir
- x509_voms_dir path - the path to the directory containing *.lsc files
needed for checking validity of VOMS extensions. If not specified default
value /etc/grid-security/vomsdir is used.
Example:
x509_voms_dir="/etc/grid-security/vomsdir" - lrms
- ARC supports various LRMS flavours, as listed in this section. For
detailed description of options please refer to ARC CE sysadmin guide:
http://www.nordugrid.org/documents/arc-ce-sysadm-guide.pdf
ONLY ONE LRMS IS ALLOWED. MULTIPLE lrms ENTRIES WILL TRIGGER UNEXPECTED BEHAVIOUR.
lrms sets the type of the Local Resource Management System (queue system), and optionally - the default queue name, separated with a blank space: lrmstype queue_name. For lrmstype, the following systems are supported and can be chosen (one per server): fork - simple forking of jobs to the same node as the server sge - (Sun/Oracle) Grid Engine condor - Condor pbs - PBS lsf - LSF ll - LoadLeveler slurm - SLURM dgbridge - Desktop Grid
PBS has many flavours, ARC currenly supports OpenPBS, PBSPro, ScalablePBS and Torque (the official name for ScalablePBS). There is no need to specify the flavour or the version number of the PBS, simply write 'pbs'. Similarly, there is no need to specify (Sun/Oracle) Grid Engine versions and flavours. "lrmstype" MUST be set here, it is a MANDATORY parameter!
The optional queue parameter specifies the default Grid queue of the LRMS. Jobs will be submitted to this queue if they do not specify queue name in job description. Queue name must match one of the [queue/queue_name] block labels, see below.
Example:
lrms="pbs gridlong"
lrms="pbs"
PBS options¶
- pbs_bin_path
- the path to the qstat,pbsnodes,qmgr etc PBS binaries, no need to set if
PBS is not used
Example:
pbs_bin_path="/usr/bin" - pbs_log_path
- the path of the PBS server logfiles which are used by A-REX to determine
whether a PBS job is completed. If not specified, A-REX will use qstat for
that.
Example:
pbs_log_path="/var/spool/pbs/server_logs"
Condor options¶
- condor_rank
- condor_rank - If you are not happy with the way Condor picks nodes when
running jobs, you can define your own ranking algorithm by optionally
setting the condor_rank attribute. condor_rank should be set to a ClassAd
float expression that you could use in the Rank attribute in a Condor job
description. Obviously no need to set if Condor is not used. An example:
Example:
condor_rank="(1-LoadAvg/2)*(1-LoadAvg/2)*Memory/1000*KFlops/1000000" - condor_bin_path
- condor_bin_path - Path to Condor binaries. Must be set if Condor is used.
Example:
condor_bin_path=/opt/condor/bin - condor_config
- condor_config - Path to Condor config file. Must be set if Condor is used
and the config file is not in its default location
(/etc/condor/condir_config or ~/condor/condor_config). The full path to
the file should be given.
Example:
condor_config=/opt/condor/etc/condor_config
SGE options¶
- sge_bin_path
- sge_bin_path - Path to Sun Grid Engine (SGE) binaries, MUST be set if SGE
is the LRMS used
Example:
sge_bin_path="/opt/n1ge6/bin/lx24-x86" - sge_root
- sge_root - Path to SGE installation directory. MUST be set if SGE is used.
Example:
sge_root="/opt/n1ge6" - sge_cell
- sge_cell - The name of the SGE cell to use. This option is only necessary
in case SGE is set up with a cell name different from 'default'
Example:
sge_cell="default" - sge_qmaster_port
- sge_qmaster_port, sge_execd_port - these options should be used in case
SGE command line clients require SGE_QMASTER_PORT and SGE_EXECD_PORT
environment variables to be set. Usually they are not necessary.
Example:
sge_qmaster_port="536"
sge_execd_port="537"
SLURM options¶
- slurm_bin_path
- slurm_bin_path - Path to SLURM binaries, must be set if installed outside
of normal $PATH
Example:
slurm_bin_path="/usr/bin" - slurm_wakeupperiod
- How long should infosys wait before querying SLURM for new data (seconds)
Example:
slurm_wakeupperiod="15" - slurm_use_sacct
- Should ARC use sacct instead of scontrol to get information on finished
jobs. Requires that accounting is turned on in SLURM. Default is
"no".
Example:
slurm_use_sacct="yes"
LSF options¶
- lsf_bin_path
- the PATH to LSF bin folder no need to set if LSF is not used
Example:
lsf_bin_path="/usr/local/lsf/bin/" - lsf_profile_path
- the PATH to profile.lsf no need to set if LSF is not used
Example:
lsf_profile_path="/usr/share/lsf/conf"
LL options¶
- ll_bin_path
- the PATH to the LoadLeveler bin folder no need to set if LoadLeveler is
not used
Example:
ll_bin_path="/opt/ibmll/LoadL/full/bin" - ll_consumable_resources
- support for a LoadLeveler setup using Consumable Resources no need to set
if LoadLeveler is not used
Example:
ll_consumable_resources="yes"
Desktop Grid options¶
- dgbridge_stage_dir
- Desktop Bridge www publish dir
Example:
dgbridge_stage_dir="/var/www/DGBridge" - dgbridge_stage_prepend
- Desktop Bridge URL prefix pointing to dgbridge_stage_dir
Example:
dgbridge_stage_prepend="http://edgi-bridge.example.com/DGBridge/"
Boinc options¶
- boinc_db_host boinc_db_port boinc_db_name boinc_db_user boinc_db_pass
- Connection details for the Boinc database.
Example:
boinc_db_host="localhost"
boinc_db_port="3306"
boinc_db_name="myproject"
boinc_db_user="boinc"
boinc_db_pass="password" - boinc_app_id = id
- ID of the app handled by this CE. Setting this option makes database
queries much faster in large projects with many apps.
Example:
boinc_app_id="1"
Other [common] options¶
- globus_tcp_port_range
- globus_tcp_port_range, globus_udp_port_range - Firewall configuration In a
firewalled environment the software which uses GSI needs to know what
ports are available. The full documentation can be found at:
http://dev.globus.org/wiki/FirewallHowTo These variable are similar to the
Globus environment variables: GLOBUS_TCP_PORT_RANGE and
GLOBUS_UDP_PORT_RANGE. These variables are not limited to [common], but
can be set individually for each service in corresponding section:
[grid-manager], [gridftpd] Example:
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000" - x509_user_key
- x509_user_cert, x509_user_key - Server credentials location. These
variables are similar to the GSI environment variables: X509_USER_KEY and
X509_USER_CERT These variables are not limited to [common], but can be set
individually for each service in corresponding section: [grid-manager],
[gridftpd], [nordugridmap]
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem" - x509_cert_dir
- x509_cert_dir - Location of trusted CA certificates This variable is
similar to the GSI environment variable: X509_CERT_DIR This variable is
not limited to [common], but can be set individually for each service in
corresponding section: [grid-manager], [gridftpd]
Example:
x509_cert_dir="/etc/grid-security/certificates" - gridmap
- gridmap - The gridmap file location This variable is similar to the GSI
environment variable: GRIDMAP This variable is not limited to [common],
but can be set individually for each service in corresponding section:
[grid-manager], [gridftpd] The default is /etc/grid-security/grid-mapfile
Example:
gridmap="/etc/grid-security/grid-mapfile" - voms_processing
- voms_processing - Defines how to behave if errors in VOMS AC processing
detected.
relaxed - use everything that passed validation.
standard - same as relaxed but fail if parsing errors took place and VOMS
extension is marked as critical. This is the default.
strict - fail if any parsing error was discovered.
noerrors - fail if any parsing or validation error happened. This command
can also be used in [grid-manager] and [gridftpd] blocks.
Example:
voms_processing="standard" - voms_trust_chain
- voms_trust_chain - Define the DN chain that the host services trust when
the VOMS AC from peer VOMS proxy certificate is parsed and validated.
There can be multiple "voms_trust_chain" existing, each one
corresponds to a VOMS server. This variable is similar to the information
in *.lsc file, but with two differences:
1, You don't need to create a *.lsc file per VOMS server, but create a
chain per VOMS server;
2, Regular expressions are supported when matching the DNs.
This variable is not limited to [common], but can be used in [grid-manager] and [gridftpd] blocks. This variable should be used together with voms_processing. This variable will overwrite the information in *.lsc if *.lsc exists.
Example:
voms_trust_chain = "/O=Grid/O=NorduGrid/CN=host/arthur.hep.lu.se" "/O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority"
voms_trust_chain = "/O=Grid/O=NorduGrid/CN=host/emi-arc.eu" "/O=Grid/O=NorduGrid/CN=NorduGrid Certification Authority"
voms_trust_chain = "^/O=Grid/O=NorduGrid" - enable_perflog_reporting
- enable_perflog_reporting expert-debug-on/no - Switch on or off performance
reporting. Default is no. Only switch on if you specifically need it, and
are aware of the possible local root exploit due to permissive directory.
Example:
enable_perflog_reporting="expert-debug-on" - perflogdir
- perflogdir logdir - Directory where performance logs should be stored.
Default is /var/log/arc/perflogs
Example:
perflogdir="/var/log/arc/perflogs"
[vo] block¶
[vo] block is used to define VOs and generate mapfiles from user list maintained by VO databases. VO block is a configuration block for the nordugridmap utility. Please note that [vo] block processing by nordugridmap utility depend on parameters defined in the [nordugridmap] block.[vo] block by itself does not affect authorization of client/user. For that label defined by vo="" attribute may be used in [group] block with with 'file' rule.
- id
- id blockid - specifies the unique configuration block id (this does not
affect nordugridmap utility)
Example:
id="vo_1" - vo
- vo vo_name - specifies the VO name, this name can be used in other blocks.
MUST be given.
Example:
vo="nordugrid" - file
- file path - output gridmap-file where GENERATED mapping list will be
stored. See parameters below to define how to generate this file. If the
same file specified as output for different [vo] blocks, nordugridmap will
automatically merge entries in given blocks order. Default is
'/etc/grid-security/gridmapfile'.
Example:
file="/etc/grid-security/VOs/atlas-users" - source
- source URL - the URL of the VO database which is assigned to this VO. The
nordugridmap will use this URL to automatically generate and keep
up-to-date userlist (mapfile) specified by the 'file' attribute. URL is a
multivalued attribute, several sources can be specified for the [vo] block
and all the users from those sources will be merged into the same file.
The source URLs are processed in the given order. Currently supported URL
types are:
http(s):// - URL to plain text file. File should contain a list
of DNs with optional issuer certificate authority DN
(see require_issuerdn): "user DN" ["issuer DN"]
voms(s):// - URL to VOMS-Admin interface
nordugrid - add NorduGrid VO members
ldap:// - expect LDAP-schema formatted VO Group
file:// - local file (stand-alone or dynamically generated by
nordugridmap). File should contain a list of DNs with
optional mapped unixid: "user DN" [mapped user ID]
Result of optional mapped unixid processing depend
on mapuser_processing option settings.
vo:// - reference to another [vo] configuration block
edg-mkgridmap://
- local configuration file used by edg-mkgridmap tool.
nordugridmap will parse configuration from file and
process it as additional [vo] block that will be referred
authomatically in place URL specified. This allow
easy migration from edg-mkgridmap solution without
rewriting your previous configuration (NOTE that rarely
used 'auth' directive and 'AUTO' mapping options are not
supported)
You can use either vo:// or file:// entries to specify dependencies between [vo] blocks, but using vo:// is a recommended way. For each separate source URL it is possible to override some parameters value. You can use the following syntax to perform this: source="URL < parameter1=value1 parameter2=value2"
You can override the following parameters: mapped_unixid for http(s),voms(s),ldap and file URLs cache_enable for http(s),voms(s),ldap and file URLs voms_method for voms(s) URLs mapuser_processing for file URLs with mapped_unixid='<unixid>' overrides (control mapped_unixid overriding behaviour for URL)
Example:
source="vomss://voms.ndgf.org:8443/voms/nordugrid.org"
source="vomss://lcg-voms.cern.ch:8443/voms/atlas?/atlas/Role=VO-Admin < mapped_unixid=atlasadmin"
source="vomss://kuiken.nikhef.nl:8443/voms/gin.ggf.org < voms_method=get"
source="http://www.nordugrid.org/developers.dn"
source="ldap://grid-vo.nikhef.nl/ou=lcg1,o=atlas,dc=eu-datagrid,dc=org"
source="file:///etc/grid-security/priviliged_users.dn"
source="vo://nordugrid_community"
source="nordugrid" - mapped_unixid
- mapped_unixid unixid - the local UNIXID which is used in the generated
grid-mapfile by the nordugridmap utility.
If any of the sources have already provided mapping information (file:// or vo://) behaviour depends on 'mapuser_processing' [nordugridmap] block configuration: mapuser_processing = 'overwrite': ignore already provided mapping and apply mapped_unixid for all sources mapuser_processing = 'keep': apply mapped_unixid only for sources that does not already has mapping information
[vo] block can only have one UNIXID. If 'mapped_unixid' is not specified behaviour depends on 'allow_empty_unixid' [nordugridmap] block configuration value: allow_empty_unixid = 'yes': empty value will be used for mapped_unixid which means that nordugridmap will generate only the list of DNs without mapping (consider using mapuser_processing='overwrite' along with this option or sources that does not provide previously defined mapping information) allow_empty_unixid = 'no': skip users without mapping information (if no mapping information provided by sources)
Example:
mapped_unixid="gridtest" - voms_fqan_map
- voms_fqan_map fqan unixid - the local UNIXID which is used to map voms(s)
sources with specific FQAN given. Several voms_fqan_map can be specified
for a [vo] block. For each voms(s) sources in [vo] block and every
voms_fqan_map record separate source record will be authomatically
generated with mapped_unixid overrided to specified one. Sources are
generated in a given voms_fqan_map order. Original voms(s) source URL are
processed LAST. This allows to simplify configuration, especially in
redundancy cases when several VOMS servers are used for the same VO.
Example:
voms_fqan_map="/atlas/Role=VO-Admin atlasadmin"
voms_fqan_map="/atlas/Role=production atlasprod" - require_issuerdn
- require_issuerdn yes/no - another nordugridmap option. YES would map only
those DNs obtained from the URLs which have the corresponding public CA
packages installed. Default is 'no'. Note, that some sources does not
provide issuer information (like voms(s):// or file://). If this sources
are used within [vo] block and require_issuerdn is set to 'yes' behaviour
depends on issuer_processing [nordugridmap] block configuration:
issuer_processing = 'relaxed': check only those records that have issuer
information provided, allow other sources
issuer_processing = 'strict': if issuer information was not found record
is filtered and will not be passed into mapfile
Example:
require_issuerdn="no" - filter
- filter ACL string - An ACL filter for the nordugridmap utility. Multiple
allow/deny statements are possible. The fetched DNs are filtered against
the specified rules before they are added to the generated mapfile. * can
be used as a wildcard. You may run the nordugridmap with the --test
command line option to see how the filters you specified work. If at least
one allow filter is specified implicit deny is used at the end of ACL. If
only deny filters are present - implicit allow used at the end.
Example:
filter="deny *infn*"
filter="allow *NorduGrid*"
[group] Authorisation block¶
These configuration blocks define rules used to define to which authorization group a user belongs. The group should not be mistaken for a virtual organisation (VO). A group may match a single vo if only a single check (rule) on vo membership is performed. It is however more common to allow multiple VOs in a single group. ARC also allows many other ways to assign users to groups. Technically, permissions are only granted to groups, not directly to VOs.The block specifies single authorization group. There may be multiple [group] blocks in configuration defining multiple authorization groups.
The block can be specified in two ways - either using [group/group1] like subblock declaration per group or just [group]. The two formats are equivalent. Every block (till the beginning of next block or the end of the file) defines one authorization group.
IMPORTANT: Rules in a group are processed in their order of appearance. The first matching rule decides the membership of a the user to a group and the processing STOPS. There are positively and negatively matching rules. If a rule is matched positively then the user tested is accepted into the respective group and further processing is stopped. Upon a negative match the user would be rejected for that group - processing stops too. The sign of rule is determined by prepending the rule with be omitted. A rule may also be prepended with '!' to invert result of rule, which will let the rule match the complement of users. That complement operator ('!') may be combined with the operator for positive or negative matching.
A group MUST be defined before it may be used. In this respect the arc.conf is ORDER SENSITIVE.
The authorization groups can be used in [gridftpd] and in its sub-blocks. The syntax of their specification varies with the service they are used for. For using authorization groups and VO blocks in HED framework please read "Security Framework of ARC" at http://www.nordugrid.org/documents/arc-security-documentation.pdf
- name
- name group_name - Specify name of group. If there is no such command in
block, name of subblock is used instead (that is what subblocks are used
for). For example [group/users].
Example:
name="users" - subject
- subject certificate_subject - Rule to match specific subject of user's
X.509 certificate. No masks, patterns and regular expressions are allowed.
For more information about X.509 refer to
http://www.wikipedia.org/wiki/X509
Example:
subject="/O=Grid/O=Big VO/CN=Main Boss" - file
- file path - Start reading rules from another file. That file has a bit
different format. It can't contain blocks and commands are separated from
arguments by space. Also word "subject" in subject command may
be skipped. That makes it convenient to directly add gridmap-like lists to
authorization group.
Example:
file="/etc/grid-security/local_users" - voms
- voms vo group role capabilities - Match VOMS attribute in user's
credential. Use '*' to match any value. More information about VOMS can be
found at http://grid-auth.infn.it
Example:
voms="nordugrid /nordugrid/Guests * *" - group
- group group_name [group_name ...] - Match user already belonging to one of
specified groups. Groups refered here must be defined earlier in
configuration file. Multiple group names may be specified for this rule.
That allows creating hierarchical structure of authorization groups like
Example:
group="local_admins" - plugin
- plugin timeout path [argument ...] - Run external executable or function
from shared library. Rule is matched if plugin returns 0. In arguments
following substitutions are supported:
%D - subject of certificate
%P - path to proxy
For more about plugins read documentation.
Example:
plugin="10 /opt/external/bin/permis %P" - lcas
- lcas library directory database - Call LCAS functions to check rule. Here
library is path to shared library of LCAS, either absolute or relative to
directory; directory is path to LCAS installation directory, equivalent of
LCAS_DIR variable; database is path to LCAS database, equivalent to
LCAS_DB_FILE variable. Each arguments except library is optional and may
be either skipped or replaced with ’*’.
Example:
lcas="" - remote
- remote URL ... - Check user's credentials against remote service. Only DN
groups stored at LDAP directories are supported. Multiple URLs are allowed
in this rule.
Example:
remote="ldap://grid-vo.nordugrid.org/ou=People,dc=nordugrid,dc=org" - vo
- vo vo_name ... - Match user belonging to VO specified by
"vo=vo_name" as configured in one of PREVIOUSLY defined [vo]
blocks. Multiple VO names are allowed for this rule.
Example:
vo="nordugrid" - all
- all - Matches any user identity. This command requires no arguments but
still can be written as all="" or all= for consistency.
Example:
all=""
The [grid-manager] block¶
The [grid-manager] block configures the part of A-REX service hosted in arched taking care of the grid tasks on the frontend (stagein/stageout, LRMS job submission, caching, etc..). Name of this block is historical and comes from times then this functionality was handled by separate process called grid-manager. This section also configures WS interfaces of A-REX service also hosted by same container.- controldir
- controldir path - The directory of the A-REX's internal job log files, not
needed on the nodes. <must be set>
Example:
controldir="/var/spool/nordugrid/jobstatus" - sessiondir
- sessiondir path [drain] - the directory which holds the sessiondirs of the
grid jobs. Multiple session directories may be specified by specifying
multiple sessiondir commands. In this case jobs are spread evenly over the
session directories. If sessiondir="*" is set, the session
directory will be spread over the ${HOME}/.jobs directories of every
locally mapped unix user. It is preferred to use common session
directories. The path may be followed by "drain", in which case
no new jobs will be assigned to that sessiondir, but current jobs will
still be processed and accessible. <sessiondir must be set>
Example:
sessiondir="/scratch/grid"
sessiondir="/mnt/grid drain" - runtimedir
- runtimedir path - The directory which holds the runtimeenvironment
scripts, should be available on the nodes as well! The runtimeenvironments
are automatically detected and advertised in the information system.
Example:
runtimedir="/SOFTWARE/runtime" - scratchdir
- scratchdir path - path on computing node to move session directory to
before execution. If defined should contain the path to the directory on
the computing node which can be used to store a jobs' files during
execution. Sets the environment variable RUNTIME_LOCAL_SCRATCH_DIR.
Default is not to move session directory before execution.
Example:
scratchdir="/local/scratch/" - shared_scratch
- shared_scratch path - path on frontend where scratchdir can be found. If
defined should contain the path corresponding to that set in scratchdir as
seen on the frontend machine. Sets the environment variable
RUNTIME_FRONTEND_SEES_NODE.
Example:
shared_scratch="/mnt/scratch" - nodename
- nodename path - command to obtain hostname of computing node.
Example:
nodename="/bin/hostname" - cachedir
- cachedir cache_path [link_path] - specifies a directory to store cached
data. Multiple cache directories may be specified by specifying multiple
cachedir commands. Cached data will be distributed evenly over the caches.
Specifying no cachedir command or commands with an empty path disables
caching. Optional link_path specifies the path at which the cache_path is
accessible on computing nodes, if it is different from the path on the
A-REX host. Example: cache="/shared/cache /frontend/jobcache" If
"link-path" is set to '.' files are not soft-linked, but copied
to session directory. If a cache directory needs to be drained, then
cachedir should specify "drain" as the link path, in which case
no new files will be added to the cache.
Example:
cachedir="/scratch/cache"
cachedir="/fs1/cache drain" - remotecachedir
- remotecachedir cache_path [link_path] - specifies caches which are under
the control of other A-REXs, but which this A-REX can have read-only
access to. Multiple remote cache directories may be specified by
specifying multiple remotecachedir commands. If a file is not available in
paths specified by cachedir, A-REX looks in remote caches. link_path has
the same meaning as in cachedir, but the special path ``replicate'' means
files will be replicated from remote caches to local caches when they are
requested.
Example:
remotecachedir="/mnt/fs1/cache replicate" - cachesize
- cachesize max min - specifies high and low watermarks for space used by
cache, as a percentage of the space on the file system on which the cache
directory is located. When the max is exceeded, files will be deleted to
bring the used space down to the min level. It is a good idea to have the
cache on its own separate file system. To turn off this feature
"cachesize" without parameters can be specified.
Example:
cachesize="80 70" - cachelifetime
- If cache cleaning is enabled, files accessed less recently than the given
time period will be deleted. Example values of this option are 1800, 90s,
24h, 30d. When no suffix is given the unit is seconds.
Example:
cachelifetime="30d" - cacheshared
- cacheshared yes|no - specifies whether the caches share a filesystem with
other data. If set to yes then cache-clean calculates the size of the
cache instead of using filesystem used space.
Example:
cacheshared="yes" - cachespacetool
- cachespacetool path [options] - specifies an alternative tool to
"df" that cache-clean should use to obtain space information on
the cache file system. The output of this command must be
"total_bytes used_bytes". The cache directory is passed as the
last argument to this command.
Example:
cachespacetool="/etc/getspace.sh" - cachelogfile
- cachelogfile path - specifies the filename where output of the cache-clean
tool should be logged. Defaults to /var/log/arc/cache-clean.log.
Example:
cachelogfile="/tmp/cache-clean.log" - cacheloglevel
- cacheloglevel level - specifies the level of logging by the cache-clean
tool, between 0 (FATAL) and 5 (DEBUG). Defaults to 3 (INFO).
Example:
cacheloglevel="4" - cachecleantimeout
- cachecleantimeout time - the timeout in seconds for running the
cache-clean tool. If using a large cache or slow file system this value
can be increased to allow the cleaning to complete. Defaults to 3600 (1
hour).
Example:
cachecleantimeout="10000" - cacheaccess
- cacheaccess rule - rules for allowing access to files in the cache
remotely through the A-REX web interface. A rule has three parts:
1. Regular expression defining a URL pattern
2. Credential attribute to match against a client's credential
3. Regular expression defining a credential value to match against a
client's
credential A client is allowed to access the cached file if a URL pattern
matches the cached file URL and the client's credential has the attribute
and matches the value required for that pattern. Possible values for
credential attribute are dn, voms:vo, voms:role and voms:group. Remote
cache access requires that the A-REX web interface is enabled via
arex_mount_point.
Examples:
cacheaccess="gsiftp://host.org/private/data/.* voms:vo myvo:production"
cacheaccess="gsiftp://host.org/private/data/ng/.* dn /O=Grid/O=NorduGrid/.*" - enable_cache_service
- enable_cache_service yes|no - Turn on or off the cache service interface.
If turned on the cache service must be installed and the A-REX WS
interface must be enabled via arex_mount_point. The interface is
accessible at the same host and port as given inn arex_mount_point with
path /cacheservice. Default is off.
Example:
enable_cache_service="yes" - user
- user user[:group] - Switch to a non root user/group after startup. Use
with caution.
Example:
user="grid" - debug
- debug debuglevel - Set debug level of the arched daemon hosting A-REX
service, between 0 (FATAL) and 5 (DEBUG). Defaults to 3 (INFO).
Example:
debug="2" - logfile
- logfile path - Specify log file location. If using an external log
rotation tool be careful to make sure it matches the path specified here.
Default log file is "/var/log/arc/grid-manager.log"
Example:
logfile="/var/log/arc/grid-manager.log" - wslogfile
- wslogfile path - Specify log file location for WS-interface operations.
This file is only created if the WS-interface is enabled through the
arex_mount_point option. The logsize, logreopen and debug options also
apply to this file. If using an external log rotation tool be careful to
make sure it matches the path specified here. It is possible to specify
the same file as logfile to combine the logs. Default is
/var/log/arc/ws-interface.log.
Example:
wslogfile="/var/log/arc/ws-interface.log" - logsize
- logsize size [number] - 'Size' specifies in bytes how big log file is
allowed to grow (approximately). If log file exceeds specified size it is
renamed into logfile.0. And logfile.0 is renamed into logfile.1, etc. up
to 'number' logfiles. Don't set logsize if you don't want to enable the
ARC logrotation because another logrotation tool is used.
Example:
logsize="100000 2" - logreopen
- logreopen yes|no - Specifies if log file must be closed after each record
is added. By default arched keeps log file open. This option can be used
to make behaviour of arched compatible with external log rotation
utilities.
Example:
logreopen="no" - pidfile
- pidfile path - Specify location of file containing PID of daemon process.
This is useful for automatic start/stop scripts.
Example:
pidfile="/var/run/arched-arex.pid" - gnu_time
- the gnu time command, default /usr/bin/time
Example:
gnu_time="/usr/bin/time" - shared_filesystem
- if computing node can access session directory at frontend, defaults to
'yes'
Example:
shared_filesystem="yes" - specifies the email address from where the notification mails are sent,
<must be specified>
Example:
mail="grid.support@somewhere.org" - joblog
- joblog path - specifies where to store specialized log about started and
finished jobs. If path is empty or no such command - log is not written.
This log is not used by any other part of ARC, so keep it disabled unless
needed.
Example:
joblog="/var/log/arc/gm-jobs.log" - jobreport
- jobreport [URL ...] [timeout] - tells to report all started and finished
jobs to logger service at 'URL'. Multiple URLs and multiple jobreport
commands are allowed. In that case the job info will be sent to all of
them. Timeout specifies how long (in days) to try to pass information
before give up. Suggested value is 30 days.
Example:
jobreport="https://grid.uio.no:8001/logger" - jobreport_publisher
- jobreport publisher - name of the accounting records publisher.
Example:
jobreport_publisher="jura" - jobreport_credentials
- jobreport credentials path [key_file [cert_file [ca_dir]]] - specifies the
credentials for accessing the accounting service.
Example:
jobreport_credentials="/etc/grid-security/hostkey.pem /etc/grid-security/hostcert.pem /etc/grid-security/certificates" - jobreport_options
- jobreport options [name:value, ...]- specifies additional parameters for
the jobreporter.
Example:
jobreport_options="urbatch:50,archiving:/tmp/archive,topic:/topic/global.accounting.cpu.central" - jobreport_logfile
- jobreport logfile - name of the file to store stderr of the publisher
executable.
Example:
jobreport_logfile="/var/log/arc/jura.log" - max_job_control_requests
- max_job_control_requests number - max number of simultaneously processed
job management requests over WS interface - like job submission, cancel,
status check etc. Default value is 100.
Example:
max_job_control_requests="100" - max_infosys_requests
- max_infosys_requests number - max number of simultaneously processed
resource info requests over WS interface. Default value is 1.
Example:
max_infosys_requests="1" - max_data_transfer_requests
- max_data_transfer_requests number - max number of simultaneously processed
data transfer requests over WS interface - like data staging. Default
value is 100.
Example:
max_data_transfer_requests="100" - maxjobs
- maxjobs number1 number2 number3 number4 - specifies maximum allowed number
of jobs.
number1 - jobs which are not in FINISHED state (jobs tracked in RAM)
number2 - jobs being run (SUBMITTING, INLRMS states)
number3 - jobs being processed per DN
number4 - jobs in whole system
number5 - LRMS scripts limit (jobs in SUBMITTING and CANCELING)
Missing number or -1 means no limit.
Example:
maxjobs="10000 10 2000" - wakeupperiod
- wakeupperiod time - specifies how often A-REX cheks for new jobs arrived,
job state change requests, etc. That is resposivity of A-REX. 'time' is
time period in seconds. Default is 3 minutes. Usually this command is not
needed because important state changes are also trigering out-of-schedule
checks.
NOTE: This parameter does not affect responsivity of backend scripts - especially scan-*-job. That means that upper estimation of time for detecting job finished executing is sum of responsivity of backend script + wakeupperiod.
Example:
wakeupperiod="180" - defaultttl
- defaultttl [ttl [ttr]] - ttl is the time in seconds for how long a session
directory will survive after job execution has finished. If not specified
the default is 1 week. ttr is how long information about a job will be
kept after the session directory is deleted. If not specified, the ttr
default is one month.
Example:
defaultttl="259200" - authplugin
- authplugin state options plugin_path - Every time job goes to 'state' run
'plugin_path' executable. Options consist of key=value pairs separated by
','.
Possible keys are timeout - wait for result no longer that 'value' seconds (timeout= can be omitted). onsuccess,onfailure,ontimeout - what to do if plugin exited with exit code 0, not 0, timeout achieved. Possible actions are: pass - continue executing job, fail - cancel job, log - write to log fail about problem and continue executing job.
Example:
authplugin="ACCEPTED timeout=10 /usr/libexec/arc/bank %C/job.%I.local %S" - authplugin
- ARC is distributed with the plugin "inputcheck". Its purpose is
to check if input files requested in job's RSL are accessible from this
machine. It is better to run it before job enters cluster. It accepts 2
arguments: names of files containing RSL and credentials' proxy. This
plugin is only guaranteed to work for job submitted through the legacy
GridFTP interface, as this is the only interface for which credentials in
the form of proxy certificate files are guaranteed to exist.
Example:
authplugin="ACCEPTED 60 /usr/libexec/arc/inputcheck %C/job.%I.description %C/job.%I.proxy" - authplugin
- ARC is distributed with the plugin "arc-vomsac-check". Its
purpose is to enforce per-queue access policies based on VOMS attributes
present in user's proxy-certificate. Plugin should be run before job
enters the cluster. It requires 2 argments: path to job information .local
file and path to credentials file. Enforced per-queue access policies are
configured with 'ac_policy' option in the [queue/name] configuration
block.
Example:
authplugin="ACCEPTED 60 /usr/libexec/arc/arc-vomsac-check -L %C/job.%I.local -P %C/job.%I.proxy" - localcred
- localcred timeout plugin_path - Every time an external executable is run
this plugin will be called. Its purpose is to set non-unix
permissions/credentials on running tasks. Note: the process itself can
still be run under the root account. If plugin_path looks like
somename@somepath, then function 'somename' from the shared library
located at 'somepath' will be called (timeout is not effective in that
case). A-REX must be run as root to use this option. Comment it out unless
you really know what you are doing.
Example:
localcred="0 acquire@/opt/nordugrid/lib/afs.so %C/job.%I.proxy" - norootpower
- norootpower yes|no - if set to yes, all job management proccesses will
switch to mapped user's identity while accessing session directory. This
is useful if session directory is on NFS root squashing turned on. Default
is no.
Example:
norootpower="yes" - allowsubmit
- allowsubmit [group ...] - list of authorization groups of users allowed to
submit new jobs while "allownew=no" is active in jobplugin
configuration. Multiple commands are allowed.
Example:
allowsubmit="mygroup"
allowsubmit="yourgroup" - helper
- helper user executable arguments - associates an external program with
A-REX. This program will be kept running under the account of the user
specified by username. Currently only ’.’ is supported as
username, corresponding to the user running A-REX. Every time this
executable finishes it will be started again. This helper plugin mechanism
can be used as an alternative to /etc/init.d or cron to (re)start external
processes.
Example:
helper=". /usr/local/bin/myutility" - tmpdir
- tmpdir - used by the A-REX, default is /tmp
Example:
tmpdir="/tmp" - maxrerun
- maxrerun - specifies how many times job can be rerun if it failed in LRMS.
Default value is 5. This is only an upper limit, the actual rerun value is
set by the user in his xrsl.
Example:
maxrerun="5" - globus_tcp_port_range
- globus_tcp_port_range, globus_udp_port_range - Firewall configuration.
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000" - x509_user_key
- x509_user_cert, x509_user_key - Location of credentials for service. These
may be used by any module or external utility which need to contact
another service not on behalf of user who submited job.
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem" - x509_cert_dir
- x509_cert_dir - Location of trusted CA certificates
Example:
x509_cert_dir="/etc/grid-security/certificates" - http_proxy
- http_proxy - http proxy server location
Example:
http_proxy="proxy.mydomain.org:3128" - fixdirectories
- fixdirectories yes|missing|no - specifies during startup A-REX should
create all directories needed for it operation and set suitable default
permissions. If "no" is specified then A-REX does nothing to
prepare its operational environment. In case of "missing" A-REX
only creates and sets permissions for directories which are not present
yet. For "yes" all directories are created and permisisons for
all used directories are set to default safe values. Default behaviour is
as if "yes" is specified.
Example:
fixdirectories="yes" - arex_mount_point
- arex_mount_point - enables web services interfaces, including job
execution and information system. The argument is an https URL defining
the endpoint port and path:
https://<hostname>:<port>/<path>
In order to submit job a client must specify the exact published path. Make sure the chosen port is not blocked by firewall or other security rules.
Example:
arex_mount_point="https://piff.hep.lu.se:443/arex" - enable_arc_interface
- enable_arc_interface yes|no - turns on or off the ARC own WS interface
based on OGSA BES and WSRF. If enabled the interface can be accessed at
the URL specified by arex_mount_point (this option must also be
specified). Default is yes.
Example:
enable_arc_interface="yes" - enable_emies_interface
- enable_emies_interface - enable the EMI Execution Service interface. If
enabled the interface can be accessed at the URL specified in
arex_mount_point (this option must also be specified)
Example:
enable_emies_interface="yes" - arguspep_endpoint
- arguspep_endpoint - specifies URL of Argus PEPD service (by default, the
argus pepd service runs on port 8154 with path /authz) to use for
authorization and user mapping. It is worth to mention that
"requireClientCertAuthentication" (default is false) item of
pepd.ini (configuration of Argus PEPD service) is set to be 'true', then
https should be used, otherwise http is proper. If specified Argus is
contacted for every operation requested through WS interface (see
arex_mount_point).
Example:
arguspep_endpoint="https://somehost.somedomain:8154/authz" - arguspep_profile
- arguspep_profile - defines which communication profile to use while
communicationg with Argus PEPD service. Possible values are:
direct - pass all authorization attributes (only for debugging)
subject - pass only subject name of client
cream - makes A-REX pretend it is gLite CREAM service. This is
recommended profile for interoperability with gLite.
emi - new profile devloped in EMI project. This is default option.
Example:
arguspep_profile="cream" - arguspep_usermap
- arguspep_usermap - specifies either response from Argus servie may define
mapping of client to local account. Possible values are 'yes' and 'no'.
Default is 'no'. Argus is contacted after all other user mapping is
performed. Hence it can overwrite all other decisions.
Example:
arguspep_usermap="no" - arguspdp_endpoint
- arguspdp_endpoint - specifies URL of Argus PDP service (by default, the
argus pepd service runs on port 8152 with path /authz) to use for
authorization and user mapping. It is worth to mention that
"requireClientCertAuthentication" (default is false) item of
pdp.ini (configuration of Argus PDP service) is set to be 'true', then
https should be used, otherwise http is proper. If specified Argus is
contacted for every operation requested through WS interface (see
arex_mount_point).
Example:
arguspdp_endpoint="https://somehost.somedomain:8152/authz" - arguspdp_profile
- arguspdp_profile - defines which communication profile to use while
communicationg with Argus PDP service. Possible values are:
subject - pass only subject name of client
cream - makes A-REX pretend it is gLite CREAM service. This is
recommended profile for interoperability with gLite.
emi - new profile devloped in EMI project. This is default option.
Example:
arguspdp_profile="cream" - arguspdp_acceptnotapplicable
- arguspdp_accpetnotapplicable - specify if the "NotApplicable"
decision returned by Argus PDP service is treated as reason to deny
request. Default is no, which treats "NotApplicable" as reson to
deny request.
Example:
arguspdp_acceptnotapplicable="no" - watchdog
- watchdog - specifies if additinal watchdog processes is spawned to restart
main process if it is stuck or dies. Possible values are 'yes' and 'no'.
Default is 'no'.
Example:
watchdog="no" - groupcfg
- groupcfg group_name [group_name ...] - specifies authorization groups for
grid-manager to accept. The main location of this parameter is inside
[gridftpd/jobs] block. The 'groupcfg' located here is only effective if
computing service is configured without GridFTP interface and hence
[gridftpd/jobs] block is missing.
Example:
groupcfg="users" - unixmap unixgroup unixvo
- unixmap [unixname][:unixgroup] rule - more sophisticated mapping to local
account
unixgroup group rule - more sophisticated mapping to local account for specific authorization groups.
unixvo vo rule - more sophisticated mapping to local account for users belonging to specified VO.
The main location for these parameters is [gridftpd] section. If located here they are only active if computing service is configured without GridFTP interface and hence [gridftpd/jobs] block is missing. For more detailed information see section [gridftpd] and read "ARC Computing Element. System Administrator guide" manual.Example:
unixmap="nobody:nogroup all"
unixgroup="users simplepool /etc/grid-security/pool/users"
unixvo="ATLAS unixuser atlas:atlas" - allowunknown
- allowunknown yes|no - check user subject against grid-mapfile. The main
location for this parameter is [gridftpd] section. If located here it is
only active if computing service is configured without GridFTP interface
and hence [gridftpd/jobs] block is missing. For more detailed information
see section [gridftpd].
Example:
allowunknown="no" - delegationdb
- delegationdb db_name - specify which DB to use to store delegations.
Currently supported db_names are bdb and sqlite. Default is bdb.
Example:
delegationdb="bdb" - forcedefaultvoms
- forcedefaultvoms VOMS_FQAN - specify VOMS FQAN which user will be assigned
if his/her credentials contain no VOMS attributes. To assign different
values to different queues put this command into [queue] block.
Example:
forcedefaultvoms="/vo/group/subgroup"
[data-staging] block¶
[data-staging] block configures DTR data staging parameters.- debug
- debug - Log level for transfer logging in job.id.errors files, between 0
(FATAL) and 5 (DEBUG). Default is to use value set by debug option in
[grid-manager] section.
Example:
debug="4" - maxdelivery
- maxdelivery - Maximum number of concurrent file transfers, i.e. active
transfers using network bandwidth. This is the total number for the whole
system including any remote staging hosts. Default is 10.
Example:
maxdelivery="40" - maxprocessor
- maxprocessor - Maximum number of concurrent files in each pre- and post-
processing state, eg cache check or replica resolution. Default is 10.
Example:
maxprocessor="20" - maxemergency
- maxemergency - Maximum "emergency" slots which can be assigned
to transfer shares when all slots up to the limits configured by the above
two options are used by other shares. This ensures shares cannot be
blocked by others. Default is 1.
Example:
maxemergency="5" - maxprepared
- maxprepared - Maximum number of files in a prepared state, i.e. pinned on
a remote storage such as SRM for transfer. A good value is a small
multiple of maxdelivery. Default is 200.
Example:
maxprepared="250" - sharetype
- sharetype - Scheme to assign transfer shares. Possible values are dn,
voms:vo, voms:role and voms:group.
Example:
sharetype="voms:role" - definedshare
- definedshare - Defines a share with a fixed priority, different from the
default (50). Priority is an integer between 1 (lowest) and 100 (highest).
Example:
definedshare="myvo:production 80"
definedshare="myvo:student 20" - dtrlog
- dtrlog - A file in which data staging state information (for monitoring
and recovery purposes) is periodically dumped. Default is
controldir/dtrstate.log
Example:
dtrlog="/tmp/dtrstate.log" - central_logfile
- central_logfile - A file in which all data staging messages from every job
will be logged (in addition to their job.id.errors files). If this option
is not present or the path is empty the log file is not created. Note this
file is not automatically controlled by logrotate.
Example:
central_logfile="/var/log/arc/datastaging.log" - deliveryservice
- The following 4 options are used to configure multi-host data staging.
deliveryservice - URL to a data delivery service which can perform remote
data staging
Example:
deliveryservice="https://myhost.org:60003/datadeliveryservice" - localdelivery
- localdelivery - If any deliveryservice is defined, this option determines
whether local data transfer is also performed. Default is no.
Example:
localdelivery="yes" - remotesizelimit
- remotesizelimit - Lower limit on file size (in bytes) of files that remote
hosts should transfer. Can be used to increase performance by transferring
small files using local processes.
Example:
remotesizelimit="100000" - usehostcert
- usehostcert - Whether the A-REX host certificate should be used for
communication with remote hosts instead of the users' proxies. Default is
no.
Example:
usehostcert="yes" - acix_endpoint
- acix_endpoint URL - the ARC Cache Index specified here will be queried for
every input file specified in a job description and any replicas found in
sites with accessible caches will be added to the replica list of the
input file. The replicas will be tried in the order specified by
preferredpattern.
Example:
acix_endpoint="https://cacheindex.ndgf.org:6443/data/index" - securetransfer
- securetransfer yes|no - if data connection allows to choose use
secure|non-secure data transfer. Currently only works for gridftp. default
is no
Example:
securetransfer="no" - passivetransfer
- passivetransfer yes|no - If yes, gridftp transfers are passive. Setting
this option to yes can solve transfer problems caused by firewalls.
default is no
Example:
passivetransfer="no" - httpgetpartial
- httpgetpartial yes|no - If yes, HTTP GET transfers may transfer data in
chunks/parts. If no - data is always transfered in one piece. Default is
yes.
Example:
httpgetpartial="yes" - speedcontrol
- speedcontrol min_speed min_time min_average_speed max_inactivity -
specifies how slow data transfer must be to trigger error. Tranfer is
canceled if speed is below min_speed bytes per second for at least
min_time seconds, or if average rate is below min_average_speed bytes per
second, or no data was transfered for longer than max_inactivity seconds.
Value of zero turns feature off. Default is "0 300 0 300"
Example:
speedcontrol="0 300 0 300" - preferredpattern
- preferredpattern pattern - specifies a preferred pattern on which to sort
multiple replicas of an input file. It consists of one or more patterns
separated by a pipe character (|) listed in order of preference. Replicas
will be ordered by the earliest match. If the dollar character ($) is used
at the end of a pattern, the pattern will be matched to the end of the
hostname of the replica. If an exclamation mark (!) is used at the
beginning of a pattern, any replicas matching the pattern will be excluded
from the sorted replicas.
Example:
preferredpattern="srm://myhost.ac.uk|.uk$|ndgf.org$|!badhost.org$" - copyurl
- copyurl url_head local_path - specifies that URLs, starting from
'url_head' should be accessed in a different way (most probaly unix open).
The file from obtained path will be copied to the session directory. NOTE:
'local_path' can also be of URL type. you can have several copyurl lines
Example:
copyurl="gsiftp://example.org:2811/data/ gsiftp://example.org/data/"
copyurl="gsiftp://example2.org:2811/data/ gsiftp://example2.org/data/" - linkurl
- linkurl url_head local_path [node_path] - identical to 'copyurl', only
file won't be copied, but soft-link will be created. The 'local_path'
specifies the way to access the file from the gatekeeper, and is used to
check permissions. The 'node_path' specifies how the file can be accessed
from computing nodes, and will be used for soft-link creation. If
'node_path' is missing - 'local_path' will be used. you can have multiple
linkurl settings
Example:
linkurl="gsiftp://somewhere.org/data /data"
linkurl="gsiftp://example.org:2811/data/ /scratch/data/" - maxtransfertries
- maxtransfertries - the maximum number of times download and upload will be
attempted per job (retries are only performed if an error is judged to be
temporary)
Example:
maxtransfertries="10"
[gridftpd] block¶
The [gridftpd] block configures the gridftpd server- user
- user user[:group] - Switch to a non root user/group after startup
WARNING: Make sure that the certificate files are owned by the user/group specified by this option. Default value is root.
Example:
user="grid" - debug
- debug debuglevel - Set debug level of the gridftpd daemon, between 0
(FATAL) and 5 (DEBUG). Default is 3 (INFO).
Example:
debug="2" - daemon
- daemon yes|no - Whether GFS is run in daemon mode. Default is yes.
Example:
daemon="yes" - logfile
- logfile path - Set logfile location
Example:
logfile="/var/log/arc/gridftpd.log" - logsize
- logsize size [number] - 'Size' specifies in bytes how big log file is
allowed to grow (approximately). If log file exceeds specified size it is
renamed into logfile.0. And logfile.0 is renamed into logfile.1, etc. up
to 'number' logfiles. Don't set logsize if you don't want to enable the
ARC logrotation because another logrotation tool is used.
Example:
logsize="100000 2" - pidfile
- pidfile path - Specify location of file containig PID of daemon process.
This is useful for automatic star/stop scripts.
Example:
pidfile="/var/run/gridftpd.pid" - port
- port bindport - Port to listen on (default 2811)
Example:
port="2811" - pluginpath
- pluginpath - directory where the plugin libraries are installed, default
is $ARC_LOCATION/lib(64)/arc
Example:
pluginpath="/usr/lib/arc/" - encryption
- encryption yes|no - should data encryption be allowed, default is no,
encryption is very heavy
Example:
encryption="no" - include
- include - Include contents of another configuration file.
Example:
include="path" - allowunknown
- allowunknown yes|no - if no, check user subject against grid-mapfile and
reject if missing. By default unknown (not in the grid-mapfile) grid users
are rejected
Example:
allowunknown="no" - allowactivedata yes|no - if no, only passive data transfer is allowed.
- By default both passive and active data transfers are allowed.
Example
allowactivedata="yes" - maxconnections
- maxconnections - maximum number of connections accepted by a gridftpd
server. Default is 100.
Example:
maxconnections="200" - defaultbuffer
- defaultbuffer size - defines size of every buffer for data
reading/writing. Default is 65536. The actual value may decrease if the
cumulative size of all buffers exceeds value specified by maxbuffer.
Example:
defaultbuffer="65536" - maxbuffer
- maxbuffer size - defines maximal amount of memory in bytes to be allocated
for all data reading/writing buffers. Default is 640kB. The number of
buffers is (max {3, min {41, 2P + 1}}), where P is the parallelism level
requested by the client. Hence, even without parallel streams enabled
number of buffers will be 3.
Example:
maxbuffer="655360" - globus_tcp_port_range
- globus_tcp_port_range, globus_udp_port_range - Firewall configuration
Example:
globus_tcp_port_range="9000,12000"
globus_udp_port_range="9000,12000" - firewall
- firewall - hostname or IP addres to use in response to PASV command
instead of IP address of a network interface of computer.
Example:
firewall="hostname" - x509_user_key
- x509_user_cert, x509_user_key - Server credentials location
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem" - x509_cert_dir
- x509_cert_dir - Location of trusted CA certificates
Example:
x509_cert_dir="/etc/grid-security/certificates" - gridmap
- gridmap - The gridmap file location The default is
/etc/grid-security/grid-mapfile
Example:
gridmap="/etc/grid-security/grid-mapfile" - unixmap
- unixmap [unixname][:unixgroup] rule - more sophisticated way to map Grid
identity of client to local account. If client matches 'rule' it's
assigned specified unix identity or one generated by rule. Mapping
commands are processed sequentially and processing stops at first
successful one (like in [group] section). For possible rules read
"ARC Computing Element. System Administrator guide" manual. All
rules defined in [group] section canbe used. There are also additional
rules which produce not only yes/no result but also give back user and
group names to which mapping should happen. The way it works is quite
complex so it is better to read full documentation.
For safety reasons if sophisticated mapping is used it is better to finish mapping sequence with default mapping to nonexistent or safe account.
Example:
unixmap="nobody:nogroup all" - unixgroup
- unixgroup group rule - do mapping only for users belonging to specified
authorization 'group'. It is similar to an additional filter for unixmap
command which filters out all users not belonging to specified
authorization group. Only rules which generate unix user and group names
may be used in this command. Please read "ARC Computing Element
System Administrator Guide" for more information.
Example:
unixgroup="users simplepool /etc/grid-security/pool/users" - unixvo
- unixvo vo rule - do mapping only for users belonging to specified VO. Only
rules which generate unix identity name may be used in this command.
Please read "ARC Computing Element. System Administrator Guide"
for more information. This command is similar to 'unixgroup' described
above and exists for convenience for setups which base mapping on VOs
users belong to.
Example:
unixvo="ATLAS unixuser atlas:atlas"
[gridftpd/filedir] block¶
[gridftpd/filedir] "fileplugin" storage block subblock for "exporting" a directory using the gridftpd's fileplugin plugin. gridftp plugins are shared libraries. "filedir" is a unique label. The access control is set by using the "dir" configuration option- plugin
- plugin name - specifies name of shared library to be loaded relative to
"pluginpath". The next line is MUST for a gridftp file server
with "fileplugin", don't change anything
Example:
plugin="fileplugin.so" - groupcfg
- groupcfg group_name [group_name ...] - specifies authorization groups for
which this plugin is activated. In case groupcfg is not used the plugin is
loaded for every mapped grid user. Multiple names were may be specified
delimited by blank space. Group names are as specified in [group]
sections.
Example:
groupcfg="users" - path
- the name of the virtual directory served by the gridftp server, REQUIRED
the exported storage area is accessible as gsiftp://my_server/topdir.
"topdir" is just an example, call the virtual path anything you
like, even "/" is a valid choice.
Example:
path="/topdir" - mount
- the physical directory corresponding to the virtual one:
gsiftp://my_server/topdir will give access to the /scratch/grid directory
on my_server, REQUIRED
Example:
mount="/scratch/grid" - dir
- dir - this is the access control parameter, you can have several
"dir" lines controlling different directories within then same
block
dir path options - specifies access rules for accessing files in 'path' (relative to virtual and real path) and all the files and directories below. 'options' are: nouser - do not use local file system rights, only use those specifies in this line owner - check only file owner access rights group - check only group access rights other - check only "others" access rights if none of the above specified usual unix access rights are applied. read - allow reading files delete - allow deleting files append - allow appending files (does not allow creation) overwrite - allow overwriting already existing files (does not allow creation, file attributes are not changed) dirlist - allow obtaining list of the files cd - allow to make this directory current create owner:group permissions_or:permissions_and - allow creating new files. File will be owned by 'owner' and owning group will be 'group'. If '*' is used, the user/group to which connected user is mapped will be used. The permissions will be set to permissions_or & permissions_and. (second number is reserved for the future usage).
mkdir owner:group permissions_or:permissions_and - allow creating new directories.
Example:
Set permissions on mounted directory:
dir="/ nouser read cd dirlist delete create *:* 664:664 mkdir *:* 775:775"Example:
Adjust permissions on some subdirectories:
dir="/section1 nouser read mkdir *:* 700:700 cd dirlist"
dir="/section2 nouser read mkdir *:* 700:700 cd dirlist"
[gridftpd/jobs] subblock¶
[gridftpd/jobs] subblock which creates the jobsubmission interface, using the jobplugin of the gridftpd service. gridftp plugins are shared libraries. 'jobs' is a unique label.- path
- the path to the virtual gridftpd directory which is used during the job
submission. MUST be set.
Example:
path="/jobs" - plugin
- plugin name - specifies name of shared library to be loaded relative to
"pluginpath". The next line is MUST for a job submission service
via gridftpd "jobplugin", don't change anything!
Example:
plugin="jobplugin.so" - groupcfg
- groupcfg group_name [group_name ...] - specifies authorization groups for
which this plugin is activated. In case groupcfg is not used the plugin is
loaded for every mapped grid user.
Example:
groupcfg="users" - allownew
- The 'allownew' configuration parameter sets if the grid resource accepts
submission of new jobs. This parameter can be used to close down a grid.
The default is yes
Example:
allownew="yes" - remotegmdirs
- remotegmdirs controldir sessiondir - Specifies control and session
directories to which jobs can be submitted but which are under the control
of another A-REX. The corresponding controldir and sessiondir parameters
must be defined in another A-REX's configuration. Multiple remotegmdirs
can be specified.
Example:
remotegmdirs="/mnt/host1/control /mnt/host1/session" - maxjobdesc
- maxjobdesc size - specifies maximal allowed size of job description in
bytes. Default value is 5MB. If value is missing or 0 size is not limited.
Example:
maxjobdesc="5242880" - configfile
- configfile service_configuration_path - If [gridftpd] and [grid-manager]
configuration parts are located in separate files this configuration
option allows to link them. The service_configuration_path points to
configuration file containing [grid-manager] section. Use this option only
if You really know what You are doing.
Example:
configfile="/etc/arc.conf"
[infosys] block¶
[infosys] block configures the hosting environment of the Information services (Local Info Tree, Index Service, Registrations, see the Information System manual) provided by the OpenLDAP slapd server.- infosys_compat
- infosys_compat - Setting this variable will cause ARC to use the old
infoproviders. Basically, the new version uses A-REX to create LDIF while
the old version uses a BDII provider-script to do it. The new version is
required for GLUE2 output.
Example:
infosys_compat="disable" - infoproviders_timeout
- infoproviders_timeout - this only applies to new infoproviders. it changes
A-REX behaviour with respect to a single infoprovider run. Increase this
value if you have many jobs in the controldir and infoproviders need more
time to process. The value is in seconds. Default is 600 seconds.
Example:
infoproviders_timeout = "600" - debug
- debug - sets the debug level/verbosity of the startup script {0 or 1}.
Default is 0.
Example:
debug="1" - hostname
- hostname - the hostname of the machine running the slapd service will be
the bind for slapd. If not present, will be taken from the [common] block
or guessed
Example:
hostname="my.testbox" - port
- port - the port where the slapd service runs. Default infosys port is
2135.
Example:
port="2135" - slapd_loglevel
- slapd_loglevel - sets the native slapd loglevel (see man slapd). Slapd
logs via syslog. The default is set to no-logging (0) and it is
RECOMMENDED not to be changed in a production environment. Non-zero
slap_loglevel value causes serious performance decrease.
Example:
slapd_loglevel="0" - slapd_hostnamebind
- slapd_hostnamebind - may be used to set the hostname part of the network
interface to which the slapd process will bind. Most of the cases no need
to set since the hostname configuration parameter is already sufficient.
The default is empty. The example below will bind the slapd process to all
the network interfaces available on the server.
Example:
slapd_hostnamebind="*" - threads
- threads - the native slapd threads parameter, default is 32. If you run an
Index service too you should modify this value.
Example:
threads="128" - timelimit
- timelimit - the native slapd timelimit parameter. Maximum number of
seconds the slapd server will spend answering a search request. Default is
3600. You probably want a much lower value.
Example:
timelimit="1800" - idletimeout
- idletimeout - the native slapd idletimeout parameter. Maximum number of
seconds the slapd server will wait before forcibly closing idle client
connections. Its value must be larger than the value of
"timelimit" option. If not set, it defaults to timelimit + 1.
Example:
idletimeout="1800" - ldap_schema_dir
- ldap_schema_dir - allows to explicitly specify a path to the schema files.
Note that this doesn't override standard location, but adds the specified
path to the standard locations /etc/ldap and /etc/openldap. If you plan to
relocate Glue1 and GLUE2 schemas, all these should be in the same
directory that you specify here. this option does NOT apply to
nordugrid.schema file. Such file has a release dependent location. Default
is to use only standard locations described above.
Example:
ldap_schema_dir="/nfs/ldap/schema/" - oldconfsuffix
- oldconfsuffix .suffix - sets the suffix of the backup files of the
low-level slapd configuration files in case they are regenerated. Default
is ".oldconfig".
Example:
oldconfsuffix=".oldconfig" - overwrite_config
- overwrite_config yes|no - determines if the infosys startup scripts should
generate new low-level slapd configuration files. By default the low-level
configuration files are regenerated with every server startup making use
of the values specified in the arc.conf.
Example:
overwrite_config="yes" - registrationlog
- registrationlog path - specifies the logfile for the registration
processes initiated by your machine. Default is
"/var/log/arc/inforegistration.log"
Example:
registrationlog="/var/log/arc/inforegistration.log" - providerlog
- providerlog path - Specifies log file location for the information
provider scripts. The feature is only available with >= 0.5.26 tag.
Default is "/var/log/arc/infoprovider.log"
Example:
providerlog="/var/log/arc/infoprovider.log" - provider_loglevel
- provider_loglevel - loglevel for the infoprovider scripts (0-5). The
default is 1 (critical errors are logged)
Example:
provider_loglevel="2" - user
- user unix_user - the unix user running the infosys processes such as the
slapd, the registrations and infoprovider scripts. By default the
ldap-user is used, you can run it as root if you wish. In case of non-root
value you must make sure that the A-REX directories and their content are
readable by the 'user' and the 'user' has access to the full LRMS
information including jobs submitted by other users. The A-REX directories
(controldir, sessiondir runtimedir, cachedir) are specified in the
[grid-manager] block
Example:
user="root" - giis_location
- giis_location - If giis_location is not set, ARC_LOCATION will be used
instead.
Example:
giis_location="/usr/" - infosys_nordugrid
- These three variables decide which schema should be used for publishing
data. They can all be enabled at the same time. Default is to enable
nordugrid mds and disable glue. infosys_nordugrid - Enables NorduGrid
schema
Example:
infosys_nordugrid="enable" - infosys_glue12
- infosys_glue12 - Enables glue1.2/1.3 schema If infosys_glue12 is enabled,
then resource_location, resource_latitude and resource_longitude need to
be set in the [infosys/glue12] block. These variables do not have default
values. The rest of the variables defaults are showcased below.
Example:
infosys_glue12="disable" - infosys_glue2_ldap
- infosys_glue2 - Enables GLUE2 schema
Example:
infosys_glue2_ldap="disable" - infosys_glue2_ldap_showactivities
- infosys_glue2_ldap_showactivities - Enables GLUE2 ComputingActivities to
appear in the LDAP rendering they're currently disabled by default.
Example:
infosys_glue2_ldap_showactivities="disable" - infosys_glue2_service_qualitylevel
- infosys_glue2_service_qualitylevel - Allows a sysadmin to define a
different GLUE2 QualityLevel for A-REX. This can be used for operations.
default: production Allowed value is one of: "production",
"pre-production", "testing", "development"
Refer to GLUE2 documentation for the meaning of these strings.
Example:
infosys_glue2_service_qualitylevel="production" - slapd
- slapd - Configure where the slapd command is located, default is:
/usr/sbin/slapd
Example:
slapd="/usr/sbin/slapd" - slapadd
- slapadd - Configure where the slapadd command is located, default is:
/usr/sbin/slapadd
Example:
slapadd="/usr/sbin/slapadd"
BDII specific¶
Starting from 11.05, Nordugrid ARC only supports BDII5. These variables are usually automatically set by ARC, and are here mostly for debug purposes and to tweak exotic BDII5 installations. In general, a sysadmin should not set these.- bdii_debug_level
- bdii_debug_level - set the following to DEBUG to check bdii errors in
bdii-update.log useful not to enable slapd logs reducing performance
issues.
Example:
bdii_debug_level="ERROR" - provider_timeout
- provider_timeout - This variable allows a system administrator to modify
the behaviour of bdii-update. This is the time BDII waits for the scripts
generated by A-REX infoproviders to produce their output. Default is 300
seconds.
Example:
provider_timeout=300 - infosys_debug
- infosys_debug - This variable disables/enables an ldap-database containing
information about the ldap database itself on "o=infosys" it is
very useful for debugging. Default is enabled.
Example:
infosys_debug="disable"
BDII5 uses the following variables. These might change depending on BDII version. ARC sets them by inspecting distributed bdii configuration files. DO NOT CHANGE UNLESS YOU KNOW WHAT YOU'RE DOING
- bdii_location
- bdii_location - The installation directory for the BDII. Default is /usr
Example:
bdii_location="/usr" - bdii_var_dir
- bdii_var_dir - Contains BDII pid files and slapd pid files
Example:
bdii_var_dir="/var/run/arc/bdii" - bdii_log_dir
- bdii_log_dir - Contains infosys logs
Example:
bdii_log_dir="/var/log/arc/bdii" - bdii_tmp_dir
- bdii_tmp_dir - Contains provider scripts
Example:
bdii_tmp_dir="/var/tmp/arc/bdii" - bdii_lib_dir
- bdii_lib_dir - Contains slapd databases
Example:
bdii_lib_dir="/var/lib/arc/bdii" - bdii_update_pid_file
- bdii_update_pid_file, slapd_pid_file - Allows to change bdii-update and
slapd pidfiles filename and location
Example:
bdii_update_pid_file="/var/run/arc/bdii-update.pid"
slapd_pid_file="$bdii_var_dir/db/slapd.pid" - bdii_database
- bdii_database - Configure what ldap database backend should be used,
default is: bdb
Example:
bdii_database="bdb"
The following options are for tweaking only. Usually one should not configure them. They change the BDII configuration file generated by ARC. Please consult BDII manual for details.
- bdii_conf
- bdii_conf - Location of the bdii configuration file. ARC modifies the
original and sets it as default /var/run/arc/infosys/bdii.conf
Example:
bdii_conf="/var/run/arc/infosys/bdii.conf"
Command line options used to run bdii-update. ARC finds it looking into bdii configuration. default: ${bdii_location}/sbin/bdii-update
bdii_update_cmd
bdii_archive_size
bdii_db_config
bdii_breathe_time
bdii_delete_delay
bdii_read_timeout
bdii_run_dir
bindmethod
cachettl
db_archive
db_checkpoint
EGIIS-related commands¶
- giis_fifo
- giis_fifo - path to fifo used by EGIIS. default is /var/run/arc/giis-fifo
This file is automatically created by ARC, the option is only for
tweaking.
Example:
giis_fifo=/var/run/arc/giis-fifo
LDAP parameters of the cluster.pl (old) infoprovider, use the defaults, do NOT change them unless you know what you are doing
- cachetime
- cachetime affects old infoproviders, and forces the validity time of the
record.
Example:
cachetime="30" - sizelimit
- sizelimit affects registration to egiis
Example:
sizelimit="10" - slapd_cron_checkpoint
- slapd_cron_checkpoint - LDAP checkpoint enable/disable This option was
introduced to solve bug #2032, to reduce the number of log files produced
by BDII. It is usually not needed, but if BDII produces large logs and
huge number of files, should help solving the issues related to that.
Example:
slapd_cron_checkpoint="enable"
[infosys/glue12] block¶
This block holds information that is needed by the glue 1.2 generation. This is only necessary if infosys_glue12 is enabled.- resource_location
- These variables need to be set if infosys_glue12 is enabled. IMPORTANT: no
slashes or backslashes here! Example: "Kastrup, Denmark"
Example:
resource_location="" - resource_latitude
- Example: "55.75000"
Example:
resource_latitude="" - resource_longitude
- Example: "12.41670"
Example:
resource_longitude="" - cpu_scaling_reference_si00
- Example 2400
Example:
cpu_scaling_reference_si00="" - processor_other_description
- Example Cores=3,Benchmark=9.8-HEP-SPEC06
Example:
processor_other_description="" - glue_site_web
- Example http://www.ndgf.org
Example:
glue_site_web="" - glue_site_unique_id
- Example NDGF-T1
Example:
glue_site_unique_id="" - provide_glue_site_info
- This variable decides if the GlueSite should be published. In case you
want to set up a more complicated setup with several publishers of data to
a GlueSite, then you may wish to tweak this parameter.
Example:
provide_glue_site_info="true"
[infosys/site/sitename] block¶
[infosys/site/sitename] Site BDII configuration block, this block is used to configure ARC to generate a site-bdii that can be registered in GOCDB etc to make it a part of a gLite network. The sitename part is to be declarative of the site-bdii being generated.- unique_id
- The unique id used to identify this site, eg "NDGF-T1"
Example:
unique_id="" - url
- The URL is of the format:
ldap://host.domain:2170/mds-vo-name=something,o=grid and should point to
the resource-bdii
Example:
url=""
[infosys/admindomain] block¶
[infosys/admindomain] GLUE2 AdminDomain configuration block, to configure administrative items of the cluster. This values do not affect neither glue12 or nordugrid renderings. If the whole block is not specified, will default to an AdminDomain called UNDEFINEDVALUE.- name
- name - the Name attribute for the domain. This will show in top-BDII to
group the resources belonging to this cluster. to group a bunch of
clusters under the same AdminDomain, just use the same name. If not
specified, will default to UNDEFINEDVALUE.
Example:
name="ARC-TESTDOMAIN" - description
- description - description of this domain. Not mandatory.
Example:
description="ARC test Domain" - www
- www - URL pointing at a site holding information about the AdminDomain.
Not mandatory.
Example:
www="http://www.nordugrid.org/" - distributed
- distributed - set this to yes if the domain is distributed that means, if
the resources belonging to the domain are considered geographically
distributed.
Example:
distributed=yes - owner
- owner - contact email of a responsible subject for the domain
Example:
owner=admin@nordugrid.org - otherinfo
- otherinfo - fills the OtherInfo GLUE2 field. no need to set, used only for
future development.
Example:
otherinfo=Test Other info
[infosys/index/indexname] block¶
[infosys/index/indexname] Index Service block configures and enables an Information Index Service. A separate Index block is required for every Index Service you may run on the given machine. The 'indexname' constitutes to the- name
- name - The unique (within the hosting machine) name of the Index Service.
Its value becomes part of the LDAP suffix of the Index Service:
(mds-vo-name=value of the name attribute, o=grid)
Example:
name="indexname" - allowreg
- allowregistration - Implements registration filtering within an Index
Sevice Sets the Local Information Trees or lower level Index Services
allowed to register to the Index Service. List each allowed registrants
with the allowreg attribute.
WARNING: specifying allowreg implies setting up a strict filtering, only the matching registrants will be able to register to the Index. The wildcard * can be used in allowreg. Several allowreg lines can be used. Some examples: -All the Swedish machines can register regardless they are resources or Indices allowreg="*.se:2135" -Cluster resources from Denmark can register allowreg="*.dk:2135/nordugrid-cluster-name=*, Mds-Vo-name=local, o=grid" -Storage resources from HIP, Finland can register allowreg="*hip.fi:2135/nordugrid-se-name=*, Mds-Vo-name=local, o=grid" -The index1.sweden.se can register as a Sweden Index (and only as a Sweden Index) allowreg="index1.sweden.se:2135/Mds-vo-Name=Sweden,o=Grid" -Any Index Service can register allowreg="*:2135/Mds-vo-Name=*,o=Grid"
Example:
allowreg="trusted.host.org.se:2135/Mds-vo-Name=Trusted-Index,o=Grid"
[infosys/index/indexname/registration/registrationname] block¶
[infosys/index/indexname/registration/registrationname] Index service registration block This block enables a registration process initiated by the to a target Index Service. NorduGrid maintains a webpage with information on major Index Services: http://www.nordugrid.org/NorduGridMDS/index_service.html- targethostname
- targethostname - the hostname of the machine running the registration
target Index Service
Example:
targethostname="index.myinstitute.org" - targetport
- targetport - the port on which the target Index Service is running. The
default is the 2135 Infosys port.
Example:
targetport="2135" - targetsuffix
- targetsuffix - the LDAP suffix of the target Index Service
Example:
targetsuffix="mds-vo-name=BigIndex,o=grid" - regperiod
- regperiod - The registration period in seconds, the registration messages
are continously sent according to the regperiod. Default is 120 sec.
Example:
regperiod="300" - registranthostname
- registranthostname - the hostname of the machine sending the
registrations. This attribute inherits its value from the [common] and
[infosys] blocks, most cases no need to set.
Example:
registranthostname="myhost.org" - registrantport
- registrantport - the port of the slapd service hosting the registrant
Index Service. The attribute inherits its value from the [infosys] block
(and therefore defaults to 2135)
Example:
registrantport="2135" - registrantsuffix
- registrantsuffix - the LDAP suffix of the registrant Index Service. It is
automatically determined from the registration block name, therefore most
of the cases no need to specify. In this case the default registrantsuffix
will be:
"Mds-Vo-name=indexname"
please mind uppercase/lowercase characters in the above string when defining allowreg in an index! Don't set it unless you want to overwrite the default.
Example:
registrantsuffix="mds-vo-name=indexname,o=grid"
[cluster] block¶
This block configures how your cluster is seen on the grid monitor (infosys point of view). Please consult the Infosys manual for detailed information on cluster attributes. If you want your cluster (configured below) to appear in the infosys (on the monitor) you also need to create a cluster registration block (see the next block).- hostname
- hostname - the FQDN of the frontend node, if the hostname is not set
already in the common block then it MUST be set here
Example:
hostname="myhost.org" - interactive_contactstring
- interactive_contactstring - the contact string for interactive logins, set
this if the cluster supports some sort of grid-enabled interactive login
(gsi-ssh), multivalued
Example:
interactive_contactstring="gsissh://frontend.cluster:2200" - cluster_alias
- alias - an arbitrary alias name of the cluster, optional
Example:
cluster_alias="Big Blue Cluster in Nowhere" - comment
- comment - a free text field for additional comments on the cluster in a
single line, no newline character is allowed!
Example:
comment="This cluster is specially designed for XYZ applications: www.xyz.org" - cluster_location
- cluster_location - The geographical location of the cluster, preferably
specified as a postal code with a two letter country prefix
Example:
cluster_location="DK-2100" - cluster_owner
- cluster_owner - it can be used to indicate the owner of a resource,
multiple entries can be used
Example:
cluster_owner="World Grid Project"
cluster_owner="University of NeverLand" - authorizedvo
- authorizedvo - this attribute is used to advertise which VOs are
authorized on the cluster. Multiple entries are allowed. This entries will
be shown in GLUE2 AccessPolicy and MappingPolicy objects.
Example:
authorizedvo="developer.nordugrid.org"
authorizedvo="community.nordugrid.org" - clustersupport
- clustersupport - this is the support email address of the resource,
multiple entries can be used
Example:
clustersupport="grid.support@mysite.org"
clustersupport="grid.support@myproject.org" - lrmsconfig
- lrmsconfig - an optional free text field to describe the configuration of
your Local Resource Management System (batch system).
Example:
lrmsconfig="single job per processor" - homogeneity
- homogeneity - determines whether the cluster consists of identical NODES
with respect to cputype, memory, installed software (opsys). The frontend
is NOT needed to be homogeneous with the nodes. In case of inhomogeneous
nodes, try to arrange the nodes into homogeneous groups assigned to a
queue and use queue-level attributes. Possible values: True,False, the
default is True. False will trigger multiple GLUE2 ExecutionEnvironments
to be published if applicable.
Example:
homogeneity="True" - architecture
- architecture - sets the hardware architecture of the NODES. The
"architecture" is defined as the output of the "uname
-m" (e.g. i686). Use this cluster attribute if only the NODES are
homogeneous with respect to the architecture. Otherwise the queue-level
attribute may be used for inhomogeneous nodes. If the frontend's
architecture agrees to the nodes, the "adotf" (Automatically
Determine On The Frontend) can be used to request automatic determination.
Example:
architecture="adotf" - opsys
- opsys - this multivalued attribute is meant to describe the operating
system of the computing NODES. Set it to the opsys distribution of the
NODES and not the frontend! opsys can also be used to describe the kernel
or libc version in case those differ from the originally shipped ones. The
distribution name should be given as distroname-version.number, where
spaces are not allowed. Kernel version should come in the form
kernelname-version.number. If the NODES are inhomogeneous with respect to
this attribute do NOT set it on cluster level, arrange your nodes into
homogeneous groups assigned to a queue and use queue-level attributes.
Example:
opsys="Linux-2.6.18"
opsys="glibc-2.5.58"
opsys="CentOS-5.6" - nodecpu
- nodecpu - this is the cputype of the homogeneous nodes. The string is
constructed from the /proc/cpuinfo as the value of "model name"
and "@" and value of "cpu MHz". Do NOT set this
attribute on cluster level if the NODES are inhomogeneous with respect to
cputype, instead arrange the nodes into homogeneous groups assigned to a
queue and use queue-level attributes. Setting the
nodecpu="adotf" will result in Automatic Determination On The
Frontend, which should only be used if the frontend has the same cputype
as the homogeneous nodes.
Example:
nodecpu="AMD Duron(tm) Processor @ 700 MHz" - nodememory
- nodememory - this is the amount of memory (specified in MB) on the node
which can be guaranteed to be available for the application. Please note
in most cases it is less than the physical memory installed in the nodes.
Do NOT set this attribute on cluster level if the NODES are inhomogeneous
with respect to their memories, instead arrange the nodes into homogeneous
groups assigned to a queue and use queue-level attributes.
Example:
nodememory="512" - defaultmemory
- defaultmemory - If a user submits a job without specifying how much memory
should be used, this value will be taken first. The order is: xrsl ->
defaultmemory -> nodememory -> 1GB. This is the amount of memory
(specified in MB) that a job will request(per rank).
Example:
defaultmemory="512" - benchmark
- benchmark name value - this optional multivalued attribute can be used to
specify benchmark results on the cluster level. Use this cluster attribute
if only the NODES are homogeneous with respect to the benchmark
performance. Otherwise the similar queue-level attribute should be used.
Please try to use one of standard benchmark names given below if possible.
Example:
benchmark="SPECINT2000 222"
benchmark="SPECFP2000 333" - middleware
- middleware - the multivalued attribute shows the installed grid software
on the cluster, nordugrid and globus-ng is automatically set, no need to
specify middleware=nordugrid or middleware=globus
Example:
middleware="my grid software" - nodeaccess
- nodeaccess - determines how the nodes can connect to the internet. Not
setting anything means the nodes are sitting on a private isolated
network. "outbound" access means the nodes can connect to the
outside world while "inbound" access means the nodes can be
connected from outside. inbound & outbound access together means the
nodes are sitting on a fully open network.
Example:
nodeaccess="inbound"
nodeaccess="outbound" - dedicated_node_string
- dedicated_node_string - the string which is used in the PBS node
configuration to distinguish the grid nodes from the rest. Suppose only a
subset of nodes are available for grid jobs, and these nodes have a common
"node property" string, this case the dedicated_node_string
should be set to this value and only the nodes with the corresponding
"pbs node property" are counted as grid enabled nodes. Setting
the dedicated_node_string to the value of the "pbs node
property" of the grid-enabled nodes will influence how the totalcpus,
user freecpus is calculated. You don't need to set this attribute if your
cluster is fully available for the grid and your cluster's PBS
configuration does not use the "node property" method to assign
certain nodes to grid queues. You shouldn't use this configuration option
unless you make sure your PBS configuration makes use of the above
described setup.
Example:
dedicated_node_string="gridnode" - localse
- localse - this multivalued parameter tells the BROKER that certain URLs
(and locations below that) should be considered "locally"
available to the cluster.
Example:
localse="gsiftp://my.storage/data1/"
localse="gsiftp://my.storage/data2/" - gm_mount_point
- gm_mount_point - this is the same as the "path" from the
[gridftpd/jobs] block. The default is "/jobs". Will be cleaned
up later, do NOT touch it.
Example:
gm_mount_point="/jobs" - gm_port
- gm_port - this is the same as the "port" from the [gridftpd]
block. The default is "2811". Will be cleaned up later.
Example:
gm_port="2811" - cpudistribution
- cpudistribution - this is the CPU distribution over nodes given in the
form: ncpu:m where
n is the number of CPUs per machine
m is the number of such computers
Example: 1cpu:3,2cpu:4,4cpu:1 represents a cluster with 3 single CPU machines, 4 dual CPU machines, one machine with 4 CPUs. This command is needed to tweak and overwrite the values returned by the underlying LRMS. In general there is no need to configure it.
Example:
cpudistribution=1cpu:3,2cpu:4,4cpu:1
[infosys/cluster/registration/registrationname] block¶
Computing resource (cluster) registration block configures and enables the registration process of a computing resource to an Index Service. A cluster can register to several Index Services this case each registration process should have its own block. NorduGrid maintains a webpage with information on major Index Services: http://www.nordugrid.org/NorduGridMDS/index_service.html- targethostname
- targethostname - see description earlier
Example:
targethostname="index.myinstitute.org" - targetport
- targetport - see description earlier
Example:
targetport="2135" - targetsuffix
- targetsuffix - see description earlier
Example:
targetsuffix="mds-vo-name=BigIndex,o=grid" - regperiod
- regperiod - see description earlier
Example:
regperiod="300" - registranthostname
- registranthostname - see description earlier
Example:
registranthostname="myhost.org" - registrantport
- registrantport - see description earlier
Example:
registrantport="2135" - registrantsuffix
- registrantsuffix - the LDAP suffix of the registrant cluster resource It
is automatically determined from the [infosys] block and the registration
blockname. In this case the default registrantsuffix will be:
"nordugrid-cluster-name=hostname,Mds-Vo-name=local,o=Grid"
please mind uppercase/lowercase characters above if defining allowreg in an index! Don't set it unless you want to overwrite the default.
Example:
registrantsuffix="nordugrid-cluster-name=myhost.org,Mds-Vo-name=local,o=grid"
[queue/queue_name] block¶
Each grid-enabled queue should have a separate queue block. The queuename should be used as a label in the block name. A queue can represent a PBS/LSF/SGE/SLURM/LL queue, a SGE pool, a Condor pool or a single machine in case 'fork' type of LRMS is specified in the [common] block.Queues don't need to be registered (there is no queue registration block), once you configured your cluster to register to an Index Service the queue entries (configured with this block) automatically will be there. Please consult the ARC Information System manual for detailed information on queue attributes: http://www.nordugrid.org/documents/arc_infosys.pdf
use the queue_name for labeling the block. The special name 'fork' should be used for labeling the queue block in case you specified 'fork' type of LRMS in the [common] block.
- name
- name sets the name of the grid-enabled queue. It MUST match the queue_name
label of the corresponding queue block, see above. Use "fork" if
you specified 'fork' type of LRMS in the [common] block. Queue name MUST
be specified, even if the queue block is already correctly labeled.
Example:
name="gridlong" - homogeneity
- homogeneity - determines whether the queue consists of identical NODES
with respect to cputype, memory, installed software (opsys). In case of
inhomogeneous nodes, try to arrange the nodes into homogeneous groups and
assigned them to a queue. Possible values: True,False, the default is
True.
Example:
homogeneity="True" - scheduling_policy
- scheduling_policy - this optional parameter tells the schedulling policy
of the queue, PBS by default offers the FIFO scheduller, many sites run
the MAUI. At the moment FIFO & MAUI is supported. If you have a MAUI
scheduller you should specify the "MAUI" value since it modifies
the way the queue resources are calculated. BY default the
"FIFO" sceduller is assumed.
Example:
scheduling_policy="FIFO" - comment
- comment - a free text field for additional comments on the queue in a
single line, no newline character is allowed!
Example:
comment="This queue is nothing more than a condor pool" - maui_bin_path
- maui_bin_path - set this parameter for the path of the maui commands like
showbf in case you specified the "MAUI" scheduling_policy above.
This parameter can be set in the [common] block as well.
Example:
maui_bin_path="/usr/local/bin" - queue_node_string
- queue_node_string - In PBS you can assign nodes to a queue (or a queue to
nodes) by using the "node property" PBS node configuration
method and asssigning the marked nodes to the queue (setting the
resources_default.neednodes = queue_node_string for that queue). This
parameter should contain the "node property" string of the
queue-assigned nodes. Setting the queue_node_string changes how the
queue-totalcpus, user freecpus are determined for this queue. Essentially,
queue_node_string value is used to construct nodes= string in PBS script,
such as nodes=count:queue_node_string where count is taken from the job
description (1 if not specified). You shouldn't use this option unless you
are sure that your PBS configuration makes use of the above configuration.
Read NorduGrid PBS instructions for more information:
http://www.nordugrid.org/documents/pbs-config.html
Example:
queue_node_string="gridlong_nodes"
queue_node_string="ppn=4:ib" - sge_jobopts
- sge_jobopts - additional SGE options to be used when submitting jobs to
SGE from this queue. If in doubt, leave it commented out
Example:
sge_jobopts="-P atlas -r yes" - condor_requirements
- condor_requirements - only needed if using Condor. It needs to be defined
for each queue. Use this option to determine which nodes belong to the
current queue. The value of 'condor_requirements' must be a valid
constraints string which is recognized by a condor_status -constraint
'....' command. It can reference pre-defined ClassAd attributes (like
Memory, Opsys, Arch, HasJava, etc) but also custom ClassAd attributes. To
define a custom attribute on a condor node, just add two lines like the
ones below in the `hostname`.local config file on the node:
NORDUGRID_RESOURCE=TRUE
STARTD_EXPRS = NORDUGRID_RESOURCE, $(STARTD_EXPRS)
A job submitted to this queue is allowed to run on any node which satisfies the 'condor_requirements' constraint. If 'condor_requirements' is not set, jobs will be allowed to run on any of the nodes in the pool. When configuring multiple queues, you can differentiate them based on memory size or disk space, for example:
Example:
condor_requirements="(OpSys == "linux" && NORDUGRID_RESOURCE && Memory >= 1000 && Memory < 2000)" - lsf_architecture
- CPU architecture to request when submitting jobs to LSF. Use only if you
know what you are doing.
Example:
lsf_architecture="PowerPC" - totalcpus
- totalcpus - manually sets the number of cpus assigned to the queue. No
need to specify the parameter in case the queue_node_string method was
used to assign nodes to the queue (this case it is dynamically calculated
and the static value is overwritten) or when the queue have access to the
entire cluster (this case the cluster level totalcpus is the relevant
parameter). Use this static parameter only if some special method is
applied to assign a subset of totalcpus to the queue.
Example:
totalcpus="32" - nodecpu
- queue-level configuration parameters: nodecpu, nodememory, architecture,
opsys and benchmark should be set if they are homogeneous over the nodes
assigned to the queue AND they are different from the cluster-level value.
Their meanings are described in the cluster block. Usage: this queue
collects nodes with "nodememory=512" while another queue has
nodes with "nodememory=256" -> don't set the cluster
attributes but use the queue-level attributes. When the frontend's
architecture or cputype agrees with the queue nodes, the "adotf"
(Automatically Determine On The Frontend) can be used to request automatic
determination of architecture or nodecpu.
Example:
nodecpu="adotf"
nodememory="512"
architecture="adotf"
opsys="Fedora 16"
opsys="Linux-3.0"
benchmark="SPECINT2000 222"
benchmark="SPECFP2000 333" - ac_policy
- queue access policy rules based on VOMS attributes in user's proxy
certificate (requires the arc-vomsac-check plugin to be enabled). Matching
rules have the following format:
ac_policy="[+/-]VOMS: <FQAN>"
Please read arc-vomsac-check manual page for more information.
Example:
ac_policy="-VOMS: /badvo"
ac_policy="VOMS: /.*/Role=production" - authorizedvo
- authorizedvo - this attribute is used to advertise which VOs are
authorized on the specific queue. Multiple entries are allowed. This
entries will be shown in the MappingPolicy objects. If something is
already defined in the [cluster] block, the shown VOs will be the union
set of those defined in [cluster] with those specific to this [queue]
block.
Example:
authorizedvo="LocalUsers"
authorizedvo="atlas"
authorizedvo="community.nordugrid.org" - cachetime
- this affects old infoproviders, and forces the validity time of the
record.
Example:
cachetime="30" - sizelimit
- sizelimit affects registration to EGIIS
Example:
sizelimit="5000"
[registration/emir] block¶
Services registration into EMIR block configures and enables the registration process of a services enabled in this configuration file into EMI indexing service (EMIR).- emirurls
- List of URL separated by comma of EMIR services which are to accept
registration. This is mandatory.
Example:
emirurls="https://somehost:60002/emir" - validity
- Time in seconds for which registration records should stay valid.
Example:
validity=600 - period
- Time in seconds how othen registration record should be sent to the
registration service.
Example:
period=60 - disablereg_xbes
- disablereg_xbes may be used to selectively disable registration of A-REX
service. Possible values are yes and no. Default is no,
Example:
disablereg_xbes="no"
[nordugridmap] block¶
[nordugridmap] block configuration is used to fine-tune behaviour of the nordugridmap - an ARC tool used to generate grid-mapfiles. Please refer to [vo] block description to find information how to specify VO sources for mapfile generation. This section setup general VO-independent parameters.- x509_user_key
- x509_user_cert, x509_user_key - public certificate and privat key to be
used when fetching sources over TLS (https:// and vomss:// sources
retrieval rely on this parameter) If not specified, values defined in
[common] section will be used. If there is also no [common] section,
X509_USER_{CERT,KEY} variables are used. Default is
'/etc/grid-security/host{cert,key}.pem'
Example:
x509_user_key="/etc/grid-security/hostkey.pem"
x509_user_cert="/etc/grid-security/hostcert.pem" - x509_cert_dir
- x509_cert_dir - the directory containing CA certificates. This information
is needed by the 'require_issuerdn' [vo] block option. Default is
'/etc/grid-security/certificates/'.
Example:
x509_cert_dir="/etc/grid-security/certificates/" - generate_vomapfile
- generate_vomapfile - control is nordugridmap will generate vo-mapfile used
by arc-ur-logger. Default is 'yes'.
Example:
generate_vomapfile="yes" - vomapfile
- vomapfile - path to vo-mapfile location. Default is
/etc/grid-security/grid-vo-mapfile
Example:
vomapfile="/etc/grid-security/grid-vo-mapfile" - log_to_file
- log_to_file - control whether logging output of nordugridmap will be saved
to file. Default is 'no' (STDERR is used).
Example:
log_to_file="yes" - logfile
- logfile - specify the nordugridmap log file location when in use. Default
is '/var/log/arc/nordugridmap.log'.
Example:
logfile="/var/log/arc/nordugridmap.log" - cache_enable
- cache_enable - control whether caching of external sources will be used.
Default is 'yes'.
Example:
cache_enable="yes" - cachedir
- cachedir - specify the path where cached sources will be stored. Default
is '/var/spool/nordugrid/gridmapcache/'
Example:
cachedir="/var/spool/nordugrid/gridmapcache/" - cachetime
- cachetime - controls how many time (in seconds) cached information remains
valid. Default is 259200 (3 days).
Example:
cachetime="259200" - issuer_processing
- issuer_processing - control the behaviour of [vo] block require_issuerdn
parameter. Valid values are 'relaxed' and 'strict'. Please see
'require_issuerdn' description in [vo] block for details. Default is
'relaxed'.
Example:
issuer_processing="relaxed" - mapuser_processing
- mapuser_processing - control the behaviour of [vo] block mapped_unixid
parameter usage. Valid values are 'overwrite' and 'keep'. Please see
'mapped_unixid' description in [vo] block for details. Default is 'keep'.
Example:
mapuser_processing="keep" - allow_empty_unixid
- allow_empty_unixid - control whether empty (or unspecified) Please see
'mapped_unixid' description of [vo] block for details. Default is 'no'
Example:
allow_empty_unixid="no" - voms_method
- voms_method - control how to get information from voms(s) sources. Valid
values are:
soap - call SOAP method directly using SOAP::Lite
get - use old implementation that manually parses XML response Default is
'soap'.
Example:
voms_method="soap" - debug
- debug level - controls the verbosity of nordugridmap output. Valid values
are:
0 - FATAL - only critical fatal error shown
1 - ERROR - errors, including non-critical are shown
2 - WARNING (default) - configuration errors that can be ignored
3 - INFO - processing information
4 - VERBOSE - a bit more processing information
5 - DEBUG - lot of processing information
When test run is requested (--test command line option of the nordugridmap) debug level is automatically set to 5 (DEBUG). Default is 2 (WARNING)
Example:
debug="4" - fetch_timeout
- fetch_timeout - control how many time (in seconds) nordugridmap will wait
for external sources retrieval. Default is 15.
Example:
fetch_timeout="15"
[acix/cacheserver] block¶
The cache server component of ACIX runs alongside A-REX. It periodically scans the cache directories and composes a Bloom filter of cache content which can be pulled by an ACIX index server.- hostname
- Hostname on which the cache server listens. Default is all available
interfaces.
Example:
hostname="myhost.org" - port
- Port on which the cache server listens. Default is 5443.
Example:
port="6000" - logfile
- Log file location for the cache server. Default is
/var/log/arc/acix-cache.log
Example:
logfile="/tmp/acix-cache.log" - cachedump
- Whether to make a dump of the cache contents in a file at
$TMP/ARC-ACIX/timestamp each time the cache server runs. Default is no.
Example:
cachedump="yes"
[acix/indexserver] block¶
The index server component of ACIX collects cache content filters from a set of cache servers configured in this block. The index server can be queried for the location of cached files.- cacheserver
- ACIX cache servers from which to pull information
Example:
cacheserver="https://some.host:5443/data/cache"
cacheserver="https://another.host:5443/data/cache"
[gangliarc] block¶
Gangliarc provides monitoring of ARC-specific metrics through ganglia. It can be run with zero configuration or customised with options in the [gangliarc] block.- frequency
- The period between each information gathering cycle, in seconds. Default
is 20.
Example:
frequency="30" - gmetric_exec
- Path to gmetric executable. Default is /usr/bin/gmetric.
Example:
gmetric_exec="/usr/local/bin/gmetric" - logfile
- log file of the daemon. Default is /var/log/arc/gangliarc.log.
Example:
logfile="/tmp/gangliarc.log" - pidfile
- pid file of the daemon. Default is /var/run/gangliarc.pid.
Example:
pidfile="/tmp/gangliarc.pid" - python_bin_path
- path to python executable. Default is /usr/bin/python.
Example:
python_bin_path="/usr/local/bin/python" - metrics
- the metrics to be monitored. Default is "all". metrics takes a
comma-separated list of one or more of the following metrics:
- staging -- number of tasks in different data staging states
- cache -- free cache space
- session -- free session directory space
- heartbeat -- last modification time of A-REX heartbeat
- processingjobs -- the number of jobs currently being processed by ARC
(jobs
between PREPARING and FINISHING states)
- failedjobs -- the number of failed jobs per last 100 finished
- jobstates -- number of jobs in different A-REX internal stages
- all -- all of the above metrics
Example:
metrics="all"
2018-11-26 | NorduGrid ARC 5.4.3 |