Common options¶

All drbdsetup sub-commands accept these two options

--create-device

new-resource¶

Resources are the primary objects of any DRBD configuration. A resource must be created with the new-resource command before any volumes or minor devices can be created. Connections are referenced by name.

new-minor¶

A minor is used as a synonym for replicated block device. It is represented in the /dev/ directory by a block device. It is the application's interface to the DRBD-replicated block devices. These block devices get addressed by their minor numbers on the drbdsetup commandline.

A pair of replicated block devices may have different minor numbers on the two machines. They are associated by a common volume-number. Volume numbers are local to each connection. Minor numbers are global on one node.

del-resource¶

Destroys a resource object. This is only possible if the resource has no volumes.

del-minor¶

Minors can only be destroyed if its disk is detached.

attach, disk-options¶

Attach associates device with lower_device to store its data blocks on. The -d (or --disk-size) should only be used if you wish not to use as much as possible from the backing block devices. If you do not use -d, the device is only ready for use as soon as it was connected to its peer once. (See the net command.)

With the disk-options command it is possible to change the options of a minor while it is attached.

--disk-size size

--on-io-error err_handler

--fencing fencing_policy

--disk-barrier,
--disk-flushes,
--disk-drain

--md-flushes

--max-bio-bvecs

--resync-rate rate

--resync-after minor

--al-extents extents

--al-updates {yes | no}

--c-plan-ahead plan_time,
--c-fill-target fill_target,
--c-delay-target delay_target,
--c-max-rate max_rate

--c-min-rate min_rate

-t, --disk-timeout disk_timeout

--discard-zeroes-if-aligned {yes | no}

--disable-write-same {yes | no}

--read-balancing method

--rs-discard-granularity bytes

connect, net-options¶

Connect sets up the device to listen on af:local_addr:port for incoming connections and to try to connect to af:remote_addr:port. If port is omitted, 7788 is used as default. If af is omitted ipv4 gets used. Other supported address families are ipv6, ssocks for Dolphin Interconnect Solutions' "super sockets" and sdp for Sockets Direct Protocol (Infiniband).

The net-options command allows you to change options while the connection is established.

--protocol protocol

--connect-int time

--ping-int time

--timeout val

--sndbuf-size size

--rcvbuf-size size

--ko-count count

--max-epoch-size val

--max-buffers val

--unplug-watermark val

--allow-two-primaries

--cram-hmac-alg alg

--shared-secret secret

--after-sb-0pri asb-0p-policy

--after-sb-1pri asb-1p-policy

--after-sb-2pri asb-2p-policy

--always-asbp

--rr-conflict role-resync-conflict-policy

--data-integrity-alg hash_alg

--no-tcp-cork

--ping-timeout ping_timeout

--discard-my-data

--tentative

--on-congestion congestion_policy,
--congestion-fill fill_threshold,
--congestion-extents active_extents_threshold

--verify-alg hash-alg

--csums-alg hash-alg

--use-rle

--socket-check-timeout

resource-options¶

Changes the options of the resource at runtime.

--cpu-mask cpu-mask

--on-no-data-accessible ond-policy

primary¶

Sets the device into primary role. This means that applications (e.g. a file system) may open the device for read and write access. Data written to the device in primary role are mirrored to the device in secondary role.

Normally it is not possible to set both devices of a connected DRBD device pair to primary role. By using the --allow-two-primaries option, you override this behavior and instruct DRBD to allow two primaries.

--overwrite-data-of-peer

--force

secondary¶

Brings the device into secondary role. This operation fails as long as at least one application (or file system) has opened the device.

It is possible that both devices of a connected DRBD device pair are secondary.

verify¶

This initiates on-line device verification. During on-line verification, the contents of every block on the local node are compared to those on the peer node. Device verification progress can be monitored via /proc/drbd. Any blocks whose content differs from that of the corresponding block on the peer node will be marked out-of-sync in DRBD's on-disk bitmap; they are not brought back in sync automatically. To do that, simply disconnect and reconnect the resource.

If on-line verification is already in progress (and this node is "VerifyS"), this command silently "succeeds". In this case, any start-sector (see below) will be ignored, and any stop-sector (see below) will be honored. This can be used to stop a running verify, or to update/shorten/extend the coverage of the currently running verify.

This command will fail if the device is not part of a connected device pair.

See also the notes on data integrity on the drbd.conf manpage.

--start start-sector

-S, --stop stop-sector

invalidate¶

This forces the local device of a pair of connected DRBD devices into SyncTarget state, which means that all data blocks of the device are copied over from the peer.

This command will fail if the device is not either part of a connected device pair, or disconnected Secondary.

invalidate-remote¶

This forces the local device of a pair of connected DRBD devices into SyncSource state, which means that all data blocks of the device are copied to the peer.

On a disconnected Primary device, this will set all bits in the out of sync bitmap. As a side affect this suspends updates to the on disk activity log. Updates to the on disk activity log resume automatically when necessary.

wait-connect¶

Returns as soon as the device can communicate with its partner device.

--wfc-timeout wfc_timeout,
--degr-wfc-timeout degr_wfc_timeout,
--outdated-wfc-timeout outdated_wfc_timeout,
--wait-after-sb

wait-sync¶

Returns as soon as the device leaves any synchronization into connected state. The options are the same as with the wait-connect command.

disconnect¶

Removes the information set by the net command from the device. This means that the device goes into unconnected state and will no longer listen for incoming connections.

detach¶

Removes the information set by the disk command from the device. This means that the device is detached from its backing storage device.

-f, --force

down¶

Removes all configuration information from the device and forces it back to unconfigured state.

role¶

Shows the current roles of the device and its peer, as local/peer.

state¶

Deprecated alias for "role"

cstate¶

Shows the current connection state of the device.

dstate¶

Shows the current states of the backing storage devices, as local/peer.

resize¶

This causes DRBD to reexamine the size of the device's backing storage device. To actually do online growing you need to extend the backing storages on both devices and call the resize command on one of your nodes.

The --size option can be used to online shrink the usable size of a drbd device. It's the users responsibility to make sure that a file system on the device is not truncated by that operation.

The --assume-peer-has-space allows you to resize a device which is currently not connected to the peer. Use with care, since if you do not resize the peer's disk as well, further connect attempts of the two will fail.

When the --assume-clean option is given DRBD will skip the resync of the new storage. Only do this if you know that the new storage was initialized to the same content by other means.

The options --al-stripes and --al-stripe-size-kB may be used to change the layout of the activity log online. In case of internal meta data this may invovle shrinking the user visible size at the same time (unsing the --size) or increasing the avalable space on the backing devices.

check-resize¶

To enable DRBD to detect offline resizing of backing devices this command may be used to record the current size of backing devices. The size is stored in files in /var/lib/drbd/ named drbd-minor-??.lkbd

This command is called by drbdadm resize res after drbdsetup device resize returned.

pause-sync¶

Temporarily suspend an ongoing resynchronization by setting the local pause flag. Resync only progresses if neither the local nor the remote pause flag is set. It might be desirable to postpone DRBD's resynchronization after eventual resynchronization of the backing storage's RAID setup.

resume-sync¶

Unset the local sync pause flag.

outdate¶

Mark the data on the local backing storage as outdated. An outdated device refuses to become primary. This is used in conjunction with fencing and by the peer's fence-peer handler.

show-gi¶

Displays the device's data generation identifiers verbosely.

get-gi¶

Displays the device's data generation identifiers.

show¶

Shows all available configuration information of a resource, or of all resources. Available options:

--show-defaults

suspend-io¶

This command is of no apparent use and just provided for the sake of completeness.

resume-io¶

If the fence-peer handler fails to stonith the peer node, and your fencing policy is set to resource-and-stonith, you can unfreeze IO operations with this command.

status¶

Show the status of a resource, or of all resources. The output consists of one paragraph for each configured resource. Each paragraph contains one line for each resource, followed by one line for each device, and one line for each connection. The device and connection lines are indented. The connection lines are followed by one line for each peer device; these lines are indented against the connection line.

Long lines are wrapped around at terminal width, and indented to indicate how the lines belongs together. Available options:

--verbose

--statistics

--color={always | auto | never}

For example, the non-verbose output for a resource with only one connection and only one volume could look like this:

fs-backoffice role:Primary


  disk:UpToDate


  peer role:Secondary


    replication:Established peer-disk:UpToDate
	      

With the --verbose --statistics options, the same resource could be reported as:

fs-data role:Primary suspended:no


    write-ordering:drain


  volume:0 minor:1 disk:UpToDate


      size:10616472 read:134465 written:144800 al-writes:18 bm-writes:0


      upper-pending:0 lower-pending:0 al-suspended:no blocked:no


  peer connection:Connected role:Secondary congested:no


    volume:0 replication:Established peer-disk:UpToDate resync-suspended:no


        received:122596 sent:22204 out-of-sync:0 pending:0 unacked:0
	      

events2¶

Show the current state of all configured DRBD objects, followed by all changes to the state.

The output format is meant to be human as well as machine readable. Each line starts with the event number, which is followed by an asterisk if the event continues in the next line. The second word in each line indicates the kind of event: exists for an existing object; create, destroy, and change if an object is created, destroyed, or changed; or call or response if an event handler is called or it returns. The third word indicates the object the event applies to: resource, device, connection, peer-device, helper, or a dash (-) to indicate that the current state has been dumped completely.

The remaining words identify the object and describe the state that he object is in. Available options:

--now

--statistics

events¶

Deprecated. If possible, change to the events2 subcommand instead.

Displays every state change of DRBD and all calls to helper programs. This might be used to get notified of DRBD's state changes by piping the output to another program.

--all-devices

--unfiltered

new-current-uuid¶

Generates a new current UUID and rotates all other UUID values. This has at least two use cases, namely to skip the initial sync, and to reduce network bandwidth when starting in a single node configuration and then later (re-)integrating a remote site.

Available option:

--clear-bitmap

This can be used to skip the initial sync, if you want to start from scratch. This use-case does only work on "Just Created" meta data. Necessary steps:

One obvious side-effect is that the replica is full of old garbage (unless you made them identical using other means), so any online-verify is expected to find any number of out-of-sync blocks.

You must not use this on pre-existing data! Even though it may appear to work at first glance, once you switch to the other node, your data is toast, as it never got replicated. So do not leave out the mkfs (or equivalent).

This can also be used to shorten the initial resync of a cluster where the second node is added after the first node is gone into production, by means of disk shipping. This use-case works on disconnected devices only, the device may be in primary or secondary role.

The necessary steps on the current active server are:

Now add the disk to the new secondary node, and join it to the cluster. You will get a resync of that parts that were changed since the first call to drbdsetup in step 1.