CTDBD.CONF(5) | CTDB - clustered TDB database | CTDBD.CONF(5) |
NAME¶
ctdbd.conf - CTDB daemon configuration fileDESCRIPTION¶
This file contains CTDB configuration variables that are affect the operation of CTDB. The default location of this file is /etc/ctdb/ctdbd.conf. This file is a shell script (see sh(1)) but is usually limited to simple variable assignments and shell-style comments. CTDB configuration variables are grouped into several categories below. Variables defined in this document can also be set in a distribution-specific configuration file such as /etc/sysconfig/ctdb (Red Hat) or /etc/default/ctdb (Debian). However, these files should be reserved for variables used by the initscript. A historical alternative is /etc/ctdb/sysconfig/ctdb - this is deprecated.INITSCRIPT CONFIGURATION¶
Some options must be available to the initscript so they need to be set in the distribution-specific initscript configuration, such as /etc/sysconfig/ctdb or /etc/default/ctdb. CTDB_PIDFILE= FILENAMEFILENAME is the name of the file used to contain the
process ID (PID) of the main CTDB daemon when it is running. This is passed
from the initscript to ctdbd_wrapper(1).
Default is /var/run/ctdb/ctdbd.pid. Corresponds to --pidfile.
GLOBAL CONFIGURATION¶
These options may be used in the initscripts, daemon and scripts. CTDB_BASE= DIRECTORYDIRECTORY containing CTDB scripts and configuration
files.
CTDB_VARDIR= DIRECTORY
DIRECTORY containing CTDB files that are modified at
runtime.
Defaults to /var/ctdb, unless /var/lib/ctdb already exists in which case it is
used.
DAEMON CONFIGURATION¶
Variables in this section are processed by ctdbd_wrapper(1) and are converted into command-line arguments to ctdbd(1). Correspondence with ctdbd(1) options is shown for each variable. The the documentation for the relevant options for more details. Many of these variables are also used by event scripts. CTDB_CAPABILITY_LMASTER=yes|noDefaults to yes. Corresponds to
--no-lmaster.
CTDB_CAPABILITY_RECMASTER=yes|no
Defaults to yes. Corresponds to
--no-recmaster.
CTDB_DBDIR= DIRECTORY
Defaults to CTDB_VARDIR. Corresponds to
--dbdir.
CTDB_DBDIR_PERSISTENT= DIRECTORY
Defaults to CTDB_VARDIR/persistent. Corresponds to
--dbdir-persistent.
CTDB_DBDIR_STATE= DIRECTORY
Defaults to CTDB_VARDIR/state. Corresponds to
--dbdir-state.
CTDB_DEBUGLEVEL= DEBUGLEVEL
Default is ERR (0). Corresponds to -d or
--debug.
CTDB_EVENT_SCRIPT_DIR= DIRECTORY
Default is CTDB_BASE/events.d, so usually
/etc/ctdb/events.d. Corresponds to --event-script-dir.
CTDB_LOGFILE= FILENAME
Defaults to /var/log/log.ctdb. Corresponds to
--logfile. See also CTDB_SYSLOG.
CTDB_LVS_PUBLIC_IP= IPADDR
No default. Corresponds to
"--lvs--single-public-ip IPADDR".
CTDB_NODES= FILENAME
Default is CTDB_BASE/nodes, so usually
/etc/ctdb/nodes. Corresponds to --nlist.
CTDB_NOTIFY_SCRIPT= FILENAME
No default, usually /etc/ctdb/notify.sh. Corresponds to
--notification-script.
CTDB_MAX_PERSISTENT_CHECK_ERRORS= NUM
Default 0. Corresponds to
--max-persistent-check-errors.
CTDB_PUBLIC_ADDRESSES= FILENAME
No default, usually /etc/ctdb/public_addresses.
Corresponds to --public-addresses.
CTDB_PUBLIC_INTERFACE= INTERFACE
No default. Corresponds to
--public-interface.
CTDB_RECOVERY_LOCK= FILENAME
Defaults to /some/place/on/shared/storage, which should
be change to a useful value. Corresponds to --reclock.
CTDB_SCRIPT_LOG_LEVEL= DEBUGLEVEL
Defaults to ERR (0). Corresponds to
--script-log-level.
CTDB_SOCKET= FILENAME
Defaults to /tmp/ctdb.socket. Corresponds to
--socket.
If you change this then you probably want to set this in root's enviroment
(perhaps in a file in /etc/profile.d) so that you can use the ctdb(1)
command in a straightforward manner.
CTDB_START_AS_DISABLED=yes|no
Default is no. Corresponds to
--start-as-disabled.
CTDB_START_AS_STOPPED=yes|no
Default is no. Corresponds to
--start-as-stopped.
CTDB_SYSLOG=yes|no
Default is no. Corresponds to --syslog.
CTDB_TRANSPORT=tcp|infiniband
Defaults to tcp. Corresponds to --transport.
While the following variables do not translate into daemon options they are used
by ctdbd_wrapper(1) when starting and stopping ctdbd(1).
CTDB_SHUTDOWN_TIMEOUT= NUM
NUM is the number of seconds to wait for ctdbd(1)
to shut down gracefully before giving up and killing it.
Defaults is 30.
CTDB_STARTUP_TIMEOUT= NUM
NUM is the number of seconds to wait for ctdbd(1)
complete early initialisation up to a point where it is unlikely to abort. If
ctdbd doesn't complete the "setup" event before this timeout
then it is killed.
Defaults is 10.
NETWORK CONFIGURATION¶
NAT GATEWAY¶
NAT gateway is used to configure fallback routing for nodes when they do not host any public IP addresses. For example, it allows unhealthy nodes to reliably communicate with external infrastructure. One node in a NAT gateway group will be designated as the NAT gateway master node and other (slave) nodes will be configured with fallback routes via the NAT gateway master node. For more information, see the NAT GATEWAY section in ctdb(7). CTDB_NATGW_DEFAULT_GATEWAY= IPADDRIPADDR is an alternate network gateway to use on the NAT
gateway master node. If set, a fallback default route is added via this
network gateway.
No default. Setting this variable is optional - if not set that no route is
created on the NAT gateway master node.
CTDB_NATGW_NODES= FILENAME
FILENAME contains the list of nodes that belong to the
same NAT gateway group.
File format:
No default, usually /etc/ctdb/natgw_nodes when enabled.
CTDB_NATGW_PRIVATE_NETWORK= IPADDR/MASK
IPADDR
IPADDR/MASK is the private sub-network that is internally
routed via the NAT gateway master node. This is usually the private network
that is used for node addresses.
No default.
CTDB_NATGW_PUBLIC_IFACE= IFACE
IFACE is the network interface on which the
CTDB_NATGW_PUBLIC_IP will be configured.
No default.
CTDB_NATGW_PUBLIC_IP= IPADDR/MASK
IPADDR/MASK indicates the IP address that is used for
outgoing traffic (originating from CTDB_NATGW_PRIVATE_NETWORK) on the NAT
gateway master node. This must not be a configured public IP address.
No default.
CTDB_NATGW_SLAVE_ONLY=yes|no
When set to "yes" a node can not be a NAT
gateway master node. In this case CTDB_NATGW_PUBLIC_IFACE and
CTDB_NATGW_PUBLIC_IP are optional and unused.
Default is no.
CTDB_NATGW_STATIC_ROUTES= IPADDR/MASK[@GATEWAY] ...
Each IPADDR/MASK identifies a network or host to which
NATGW should create a fallback route, instead of creating a single default
route. This can be used when there is already a default route, via an
interface that can not reach required infrastructure, that overrides the NAT
gateway default route.
If GATEWAY is specified then the corresponding route on the NATGW master node
will be via GATEWAY. Such routes are created even if
CTDB_NATGW_DEFAULT_GATEWAY is not specified. If GATEWAY is not
specified for some networks then routes are only created on the NATGW master
node for those networks if CTDB_NATGW_DEFAULT_GATEWAY is specified.
This should be used with care to avoid causing traffic to unnecessarily
double-hop through the NAT gateway master, even when a node is hosting public
IP addresses. Each specified network or host should probably have a
corresponding automatically created link route or static route to avoid this.
No default.
CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24 CTDB_NATGW_DEFAULT_GATEWAY=10.0.0.1 CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 CTDB_NATGW_PUBLIC_IFACE=eth0
CTDB_NATGW_NODES=/etc/ctdb/natgw_nodes CTDB_NATGW_PRIVATE_NETWORK=192.168.1.0/24 CTDB_NATGW_PUBLIC_IP=10.0.0.227/24 CTDB_NATGW_PUBLIC_IFACE=eth0 CTDB_NATGW_STATIC_ROUTES=10.0.0.0/24
POLICY ROUTING¶
A node running CTDB may be a component of a complex network topology. In particular, public addresses may be spread across several different networks (or VLANs) and it may not be possible to route packets from these public addresses via the system's default route. Therefore, CTDB has support for policy routing via the 13.per_ip_routing eventscript. This allows routing to be specified for packets sourced from each public address. The routes are added and removed as CTDB moves public addresses between nodes. For more information, see the POLICY ROUTING section in ctdb(7). CTDB_PER_IP_ROUTING_CONF= FILENAMEFILENAME contains elements for constructing the desired
routes for each source address.
The special FILENAME value __auto_link_local__ indicates that no
configuration file is provided and that CTDB should generate reasonable
link-local routes for each public IP address.
File format:
No default, usually /etc/ctdb/policy_routing when enabled.
CTDB_PER_IP_ROUTING_RULE_PREF= NUM
IPADDR DEST-IPADDR/MASK [GATEWAY-IPADDR]
NUM sets the priority (or preference) for the routing
rules that are added by CTDB.
This should be (strictly) greater than 0 and (strictly) less than 32766. A
priority of 100 is recommended, unless this conflicts with a priority already
in use on the system. See ip(8), for more details.
CTDB_PER_IP_ROUTING_TABLE_ID_LOW= LOW-NUM,
CTDB_PER_IP_ROUTING_TABLE_ID_HIGH= HIGH-NUM
CTDB determines a unique routing table number to use for
the routing related to each public address. LOW-NUM and HIGH-NUM indicate the
minimum and maximum routing table numbers that are used.
ip(8) uses some reserved routing table numbers below 255. Therefore,
CTDB_PER_IP_ROUTING_TABLE_ID_LOW should be (strictly) greater than 255.
CTDB uses the standard file /etc/iproute2/rt_tables to maintain a mapping
between the routing table numbers and labels. The label for a public address
ADDR will look like ctdb. addr. This means that the associated
rules and routes are easy to read (and manipulate).
No default, usually 1000 and 9000.
CTDB_PER_IP_ROUTING_CONF=/etc/ctdb/policy_routing CTDB_PER_IP_ROUTING_RULE_PREF=100 CTDB_PER_IP_ROUTING_TABLE_ID_LOW=1000 CTDB_PER_IP_ROUTING_TABLE_ID_HIGH=9000
MISCELLANEOUS NETWORK CONFIGURATION¶
CTDB_PARTIALLY_ONLINE_INTERFACES=yes|noWhether one or more offline interfaces should cause a
monitor event to fail if there are other interfaces that are up. If this is
"yes" and a node has some interfaces that are down then ctdb
status will display the node as "PARTIALLYONLINE".
Default is "no".
SERVICE CONFIGURATION¶
CTDB can be configured to manage and/or monitor various NAS (and other) services via its eventscripts. In the simplest case CTDB will manage a service. This means the service will be started and stopped along with CTDB, CTDB will monitor the service and CTDB will do any required reconfiguration of the service when public IP addresses are failed over.SAMBA¶
Eventscripts49.winbind
50.samba
Should CTDB manage Samba?
Default is no.
CTDB_MANAGES_WINBIND=yes|no
Should CTDB manage Winbind?
Default is no.
CTDB_SAMBA_CHECK_PORTS= PORT-LIST
When monitoring Samba, check TCP ports in space-separated
PORT-LIST.
Default is to monitor ports that Samba is configured to listen on.
CTDB_SAMBA_SKIP_SHARE_CHECK=yes|no
As part of monitoring, should CTDB skip the check for the
existence of each directory configured as share in Samba. This may be
desirable if there is a large number of shares.
Default is no.
CTDB_SERVICE_NMB= SERVICE
Distribution specific SERVICE for managing nmbd.
Default is distribution-dependant.
CTDB_SERVICE_SMB= SERVICE
Distribution specific SERVICE for managing smbd.
Default is distribution-dependant.
CTDB_SERVICE_WINBIND= SERVICE
Distribution specific SERVICE for managing winbindd.
Default is "winbind".
NFS¶
This includes parameters for the kernel NFS server and the user-space NFS-Ganesha[1] server.60.nfs
60.ganesha
The type of cluster filesystem to use with NFS-ganesha.
Currently only "gpfs" is supported.
Default is "gpfs".
CTDB_GANESHA_REC_SUBDIR SUBDIR
SUBDIR is the name of a top-level subdirectory in the
first cluster filesystem. This subdirectory is used to allow communication
between NFS-Ganesha and the 60.ganesha script.
Default is ".ganesha".
CTDB_MANAGES_NFS=yes|no
Should CTDB manage NFS?
Default is no.
CTDB_MONITOR_NFS_THREAD_COUNT=yes|no
Whether to monitor the NFS kernel server thread count.
This works around a limitation in some NFS initscripts where some threads can be
stuck in host filesystem calls (perhaps due to slow storage), a restart
occurs, some threads don't exit, the start only adds the missing number of
threads, the stuck threads exit, and the result is a lower than expected
thread count. Note that if you must also set RPCNFSDCOUNT
(RedHat/Debian) or USE_KERNEL_NFSD_NUMBER (SUSE) in your NFS
configuration so the monitoring code knows how many threads there should be -
if neither of these are set then this option will be ignored.
Default is no.
CTDB_NFS_DUMP_STUCK_THREADS= NUM
NUM is the number of NFS kernel server threads to dump
stack traces for if some are still alive after stopping NFS during a restart.
Default is 5. Set this to 0 to disable this feature.
CTDB_NFS_SERVER_MODE=kernel|ganesha
Selects which NFS server to be managed.
This replaces the deprecated variable NFS_SERVER_MODE.
Default is "kernel".
CTDB_NFS_SKIP_KNFSD_ALIVE_CHECK=yes|no
During monitoring, should CTDB skip the rpcinfo
check that is used to see if the NFS kernel server is functional.
Default is no.
CTDB_NFS_SKIP_SHARE_CHECK=yes|no
As part of monitoring, should CTDB skip the check for the
existence of each directory exported via NFS. This may be desirable if there
is a large number of exports.
Default is no.
CTDB_RPCINFO_LOCALHOST= IPADDR|HOSTNAME
IPADDR or HOSTNAME indicates the address that
rpcinfo should connect to when doing rpcinfo check on RPC
service during monitoring. Optimally this would be "localhost".
However, this can add some performance overheads.
Default is "127.0.0.1".
CTDB_SKIP_GANESHA_NFSD_CHECK=yes|no
As part of monitoring, should CTDB skip the check for the
existence of each directory exported via NFS-Ganesha. This may be desirable if
there is a large number of exports.
Default is no.
APACHE HTTPD¶
CTDB can manage the Apache web server.41.httpd
Should CTDB manage the Apache web server?
Default is no.
CLAMAV¶
CTDB has support to manage the popular anti-virus daemon ClamAV.31.clamd
This eventscript is not enabled by default. Use ctdb enablescript to
enable it.Should CTDB manage ClamAV?
Default is no.
CTDB_CLAMD_SOCKET= FILENAME
FILENAME is the socket to monitor ClamAV.
No default.
ISCSI¶
CTDB has support for managing the Linux iSCSI tgtd service.70.iscsi
Should CTDB manage iSCSI tgtd?
Default is no.
CTDB_START_ISCSI_SCRIPTS= DIRECTORY
DIRECTORY on shared storage containing scripts to start
tgtd for each public IP address.
No default.
MULTIPATHD¶
CTDB can monitor multipath devices to ensure that active paths are available.20.multipathd
This eventscript is not enabled by default. Use ctdb enablescript to
enable it.MP-DEVICE-LIST is a list of multipath devices for CTDB to
monitor?
No default.
VSFTPD¶
CTDB can manage the vsftpd FTP server.40.vsftpd
Should CTDB manage the vsftpd FTP server?
Default is no.
SYSTEM RESOURCE MONITORING CONFIGURATION¶
CTDB can experience seemingly random (performance and other) issues if system resources become too contrained. Options in this section can be enabled to allow certain system resources to be checked.00.ctdb
40.fs_use
Filesystem usage monitoring is in 40.fs_use. This eventscript is not enabled by
default. Use ctdb enablescript to enable it.FS-LIMIT-LIST is a space-separated list of
FILESYSTEM: LIMIT pairs indicating that a node should be flagged
unhealthy if the space used on FILESYSTEM reaches LIMIT%.
No default.
Note that this feature uses the 40.fs_use eventscript, which is not enabled by
default. Use ctdb enablescript to enable it.
CTDB_CHECK_SWAP_IS_NOT_USED=yes|no
Should a warning be logged if swap space is in use.
Default is no.
CTDB_MONITOR_FREE_MEMORY= NUM
NUM is a lower limit on available system memory,
expressed in megabytes. If this is set and the amount of available memory
falls below this limit then some debug information will be logged, the node
will be disabled and then CTDB will be shut down.
No default.
CTDB_MONITOR_FREE_MEMORY_WARN= NUM
NUM is a lower limit on available system memory,
expressed in megabytes. If this is set and the amount of available memory
falls below this limit then a warning will be logged.
No default.
MISCELLANEOUS SERVICE-RELATED CONFIGURATION¶
CTDB_MANAGED_SERVICES= SERVICE-LISTSERVICE-LIST is a space-separated list of SERVICEs that
CTDB should manage. This can be used as an alternative to the
CTDB_MANAGES_ SERVICE variables.
No default.
CTDB_SERVICE_AUTOSTARTSTOP=yes|no
When CTDB should start and stop services if they become
managed or unmanaged.
Default is no.
TUNABLES CONFIGURATION¶
CTDB tunables (see ctdbd-tunables(7)) can be set from the configuration file. They are set as follows:CTDB_SET_ TUNABLE=VALUE
CTDB_SET_MonitorInterval=20
DEBUG AND TEST¶
Variable in this section are for debugging and testing CTDB. They should not generally be needed. CTDB_DEBUG_HUNG_SCRIPT= FILENAMEFILENAME is a script to run to log debug information when
an event script times out.
Default is CTDB_BASE/debug-hung-script.sh.
CTDB_DEBUG_HUNG_SCRIPT_LOGFILE= FILENAME
FILENAME specifies where log messages should go when
debugging hung eventscripts. This is a testing option. See also
CTDB_DEBUG_HUNG_SCRIPT.
No default. Messages go to stdout/stderr and are logged to the same place as
other CTDB log messages.
CTDB_DEBUG_HUNG_SCRIPT_STACKPAT= REGEXP
REGEXP specifies interesting processes for which stack
traces should be logged when debugging hung eventscripts and those processes
are matched in pstree output. See also CTDB_DEBUG_HUNG_SCRIPT.
Default is "exportfs\|rpcinfo".
CTDB_DEBUG_LOCKS= FILENAME
FILENAME is a script to run to log debug information when
an CTDB fails to freeze databases during recovery.
No default, usually CTDB_BASE/debug_locks.sh.
CTDB_ETCDIR= DIRECTORY
DIRECTORY containing system configuration files. This is
used to provide alternate configuration when testing and should not need to be
changed from the default.
Default is /etc.
CTDB_INIT_STYLE=debian|redhat|suse
This is the init style used by the Linux distribution (or
other operating system) being used. This is usually determined dynamically by
checking the system. This variable is used by the initscript to determine
which init system primitives to use. It is also used by some eventscripts to
choose the name of initscripts for certain services, since these can vary
between distributions.
No fixed default.
If this option needs to be changed from the calculated default for the
initscript to function properly, then it must be set in the
distribution-specific initscript configuration, such as
/etc/sysconfig/ctdb
CTDB_MAX_CORRUPT_DB_BACKUPS= NUM
NUM is the maximum number of volatile TDB database
backups to be kept (for each database) when a corrupt database is found during
startup. Volatile TDBs are zeroed during startup so backups are needed to
debug any corruption that occurs before a restart.
Default is 10.
CTDB_RC_LOCAL= FILENAME
FILENAME is a script fragment to be sourced by the
functions that is sourced by scripts. On example use would be to override
function definitions in unit tests. As a sanity check, this file must be
executable for it to be used.
No default.
CTDB_RUN_TIMEOUT_MONITOR=yes|no
Whether CTDB should simulate timing out monitor events.
This uses the 99.timeout eventscript.
Default is no.
CTDB_SCRIPT_DEBUGLEVEL= NUM
NUM is the level debugging messages printed by CTDB
scripts. Setting this to a higher number (e.g. 4) will cause some scripts to
log more messages.
Default is 2.
CTDB_SUPPRESS_COREFILE=yes|no
Whether CTDB core files should be suppressed.
Default is no.
CTDB_VALGRIND=yes|no| COMMAND
If "yes", this causes ctdbd(1) to be run
under valgrind(1) with logs going to /var/log/ctdb_valgrind. If neither
"yes" nor "no" then the value is assumed to be a COMMAND
(e.g. a valgrind variation, a gdb(1) command) that is used in
place of the default valgrind command. In either case, the
--valgrind option is passed to ctdbd.
Default is no.
FILES¶
/etc/ctdb/ctdbd.conf
/etc/sysconfig/ctdb
/etc/default/ctdb
/etc/ctdb/sysconfig/ctdb
SEE ALSO¶
ctdbd(1), ctdbd_wrapper(1), onnode(1), ctdb(7), ctdb-tunables(7), http://ctdb.samba.org/AUTHOR¶
This documentation was written by Amitay Isaacs, Martin SchwenkeCOPYRIGHT¶
Copyright © 2007 Andrew Tridgell, Ronnie SahlbergNOTES¶
- 1.
- NFS-Ganesha
11/12/2017 | ctdb |