NAME¶
locking —
kernel synchronization
primitives
DESCRIPTION¶
The
FreeBSD kernel is written to run across multiple CPUs and
as such requires several different synchronization primitives to allow the
developers to safely access and manipulate the many data types required.
Mutexes¶
Mutexes (also called "sleep mutexes") are the most commonly used
synchronization primitive in the kernel. Thread acquires (locks) a mutex
before accessing data shared with other threads (including interrupt threads),
and releases (unlocks) it afterwards. If the mutex cannot be acquired, the
thread requesting it will sleep. Mutexes fully support priority propagation.
See
mutex(9) for details.
Spin mutexes¶
Spin mutexes are variation of basic mutexes; the main difference between the two
is that spin mutexes never sleep - instead, they spin, waiting for the thread
holding the lock, which runs on another CPU, to release it. Differently from
ordinary mutex, spin mutexes disable interrupts when acquired. Since disabling
interrupts is expensive, they are also generally slower. Spin mutexes should
be used only when neccessary, e.g. to protect data shared with interrupt
filter code (see
bus_setup_intr(9) for details).
Pool mutexes¶
With most synchronisaton primitives, such as mutexes, programmer must provide a
piece of allocated memory to hold the primitive. For example, a mutex may be
embedded inside the structure it protects. Pool mutex is a variant of mutex
without this requirement - to lock or unlock a pool mutex, one uses address of
the structure being protected with it, not the mutex itself. Pool mutexes are
seldom used.
See
mtx_pool(9) for details.
Reader/writer locks¶
Reader/writer locks allow shared access to protected data by multiple threads,
or exclusive access by a single thread. The threads with shared access are
known as
readers since they should only read the protected
data. A thread with exclusive access is known as a
writer
since it may modify protected data.
Reader/writer locks can be treated as mutexes (see above and
mutex(9)) with shared/exclusive semantics. More
specifically, regular mutexes can be considered to be equivalent to a
write-lock on an
rw_lock. The
rw_lock
locks have priority propagation like mutexes, but priority can be propagated
only to an exclusive holder. This limitation comes from the fact that shared
owners are anonymous. Another important property is that shared holders of
rw_lock can recurse, but exclusive locks are not allowed to
recurse. This ability should not be used lightly and
may go
away.
See
rwlock(9) for details.
Read-mostly locks¶
Mostly reader locks are similar to
reader/writer locks but
optimized for very infrequent write locking.
Read-mostly
locks implement full priority propagation by tracking shared owners using a
caller-supplied
tracker data structure.
See
rmlock(9) for details.
Shared/exclusive locks¶
Shared/exclusive locks are similar to reader/writer locks; the main difference
between them is that shared/exclusive locks may be held during unbounded sleep
(and may thus perform an unbounded sleep). They are inherently less efficient
than mutexes, reader/writer locks and read-mostly locks. They don't support
priority propagation. They should be considered to be closely related to
sleep(9). In fact it could in some cases be considered a
conditional sleep.
See
sx(9) for details.
Counting semaphores¶
Counting semaphores provide a mechanism for synchronizing access to a pool of
resources. Unlike mutexes, semaphores do not have the concept of an owner, so
they can be useful in situations where one thread needs to acquire a resource,
and another thread needs to release it. They are largely deprecated.
See
sema(9) for details.
Condition variables¶
Condition variables are used in conjunction with mutexes to wait for conditions
to occur. A thread must hold the mutex before calling the
cv_wait*(), functions. When a thread waits on a condition,
the mutex is atomically released before the thread is blocked, then reacquired
before the function call returns.
See
condvar(9) for details.
Giant¶
Giant is an instance of a mutex, with some special characteristics:
- It is recursive.
- Drivers and filesystems can request that Giant be locked
around them by not marking themselves MPSAFE. Note that infrastructure to
do this is slowly going away as non-MPSAFE drivers either became properly
locked or disappear.
- Giant must be locked first before other locks.
- It is OK to hold Giant while performing unbounded sleep;
in such case, Giant will be dropped before sleeping and picked up after
wakeup.
- There are places in the kernel that drop Giant and pick it
back up again. Sleep locks will do this before sleeping. Parts of the
network or VM code may do this as well, depending on the setting of a
sysctl. This means that you cannot count on Giant keeping other code from
running if your code sleeps, even if you want it to.
Sleep/wakeup¶
The functions
tsleep(),
msleep(),
msleep_spin(),
pause(),
wakeup(), and
wakeup_one() handle
event-based thread blocking. If a thread must wait for an external event, it
is put to sleep by
tsleep(),
msleep(),
msleep_spin(), or
pause(). Threads may
also wait using one of the locking primitive sleep routines
mtx_sleep(9),
rw_sleep(9), or
sx_sleep(9).
The parameter
chan is an arbitrary address that uniquely
identifies the event on which the thread is being put to sleep. All threads
sleeping on a single
chan are woken up later by
wakeup(), often called from inside an interrupt routine, to
indicate that the resource the thread was blocking on is available now.
Several of the sleep functions including
msleep(),
msleep_spin(), and the locking primitive sleep routines
specify an additional lock parameter. The lock will be released before
sleeping and reacquired before the sleep routine returns. If
priority includes the
PDROP
flag, then the lock will not be reacquired before returning. The lock is used
to ensure that a condition can be checked atomically, and that the current
thread can be suspended without missing a change to the condition, or an
associated wakeup. In addition, all of the sleep routines will fully drop the
Giant mutex (even if recursed) while the thread is
suspended and will reacquire the
Giant mutex before the
function returns.
See
sleep(9) for details.
Lockmanager locks¶
Shared/exclusive locks, used mostly in
VFS(9), in particular
as a
vnode(9) lock. They have features other lock types
don't have, such as sleep timeout, writer starvation avoidance, draining, and
interlock mutex, but this makes them complicated to implement; for this
reason, they are deprecated.
See
lock(9) for details.
INTERACTIONS¶
The primitives interact and have a number of rules regarding how they can and
can not be combined. Many of these rules are checked using the
witness(4) code.
Bounded vs. unbounded sleep¶
The following primitives perform bounded sleep: mutexes, pool mutexes,
reader/writer locks and read-mostly locks.
The following primitives block (perform unbounded sleep): shared/exclusive
locks, counting semaphores, condition variables, sleep/wakeup and lockmanager
locks.
It is an error to do any operation that could result in any kind of sleep while
holding spin mutex.
As a general rule, it is an error to do any operation that could result in
unbounded sleep while holding any primitive from the 'bounded sleep' group.
For example, it is an error to try to acquire shared/exclusive lock while
holding mutex, or to try to allocate memory with M_WAITOK while holding
read-write lock.
As a special case, it is possible to call
sleep() or
mtx_sleep() while holding a single mutex. It will atomically
drop that mutex and reacquire it as part of waking up. This is often a bad
idea because it generally relies on the programmer having good knowledge of
all of the call graph above the place where
mtx_sleep() is
being called and assumptions the calling code has made. Because the lock gets
dropped during sleep, one one must re-test all the assumptions that were made
before, all the way up the call graph to the place where the lock was
acquired.
It is an error to do any operation that could result in any kind of sleep when
running inside an interrupt filter.
It is an error to do any operation that could result in unbounded sleep when
running inside an interrupt thread.
Interaction table¶
The following table shows what you can and can not do while holding one of the
synchronization primitives discussed:
You have: You want: |
spin mtx |
mutex |
sx |
rwlock |
rmlock |
sleep |
spin mtx |
ok-1 |
no |
no |
no |
no |
no-3 |
mutex |
ok |
ok-1 |
no |
ok |
ok |
no-3 |
sx |
ok |
ok |
ok-2 |
ok |
ok |
ok-4 |
rwlock |
ok |
ok |
no |
ok-2 |
ok |
no-3 |
rmlock |
ok |
ok |
no |
ok |
ok-2 |
no |
*1 Recursion is defined per lock. Lock order is important.
*2 Readers can recurse though writers can not. Lock order is
important.
*3 There are calls that atomically release this primitive when
going to sleep and reacquire it on wakeup (e.g.
mtx_sleep(),
rw_sleep() and
msleep_spin() ).
*4 Though one can sleep holding an sx lock, one can also use
sx_sleep() which will atomically release this primitive when
going to sleep and reacquire it on wakeup.
Context mode table¶
The next table shows what can be used in different contexts. At this time this
is a rather easy to remember table.
Context: |
spin mtx |
mutex |
sx |
rwlock |
rmlock |
sleep |
interrupt filter: |
ok |
no |
no |
no |
no |
no |
ithread: |
ok |
ok |
no |
ok |
ok |
no |
callout: |
ok |
ok |
no |
ok |
no |
no |
syscall: |
ok |
ok |
ok |
ok |
ok |
ok |
SEE ALSO¶
witness(4),
condvar(9),
lock(9),
mtx_pool(9),
mutex(9),
rmlock(9),
rwlock(9),
sema(9),
sleep(9),
sx(9),
LOCK_PROFILING(9)
HISTORY¶
These functions appeared in
BSD/OS 4.1 through
FreeBSD 7.0
BUGS¶
There are too many locking primitives to choose from.