LOCK_PROFILING —
kernel lock profiling support
options LOCK_PROFILING
The
LOCK_PROFILING kernel option adds support
for measuring and reporting lock use and contention statistics. These
statistics are collated by “acquisition point”. Acquisition
points are distinct places in the kernel source code (identified by source
file name and line number) where a lock is acquired.
For each acquisition point, the following statistics are accumulated:
- The longest time the lock was ever continuously held
after being acquired at this point.
- The total time the lock was held after being acquired at
this point.
- The total time that threads have spent waiting to
acquire the lock.
- The total number of non-recursive acquisitions.
- The total number of times the lock was already held by
another thread when this point was reached, requiring a spin or a
sleep.
- The total number of times another thread tried to
acquire the lock while it was held after having been acquired at this
point.
In addition, the average hold time and average wait time are derived from the
total hold time and total wait time respectively and the number of
acquisitions.
The
LOCK_PROFILING kernel option also adds
the following
sysctl(8) variables to control and
monitor the profiling code:
- debug.lock.prof.enable
- Enable or disable the lock profiling code. This defaults to
0 (off).
- debug.lock.prof.reset
- Reset the current lock profiling buffers.
- debug.lock.prof.stats
- The actual profiling statistics in plain text. The columns
are as follows, from left to right:
- max
- The longest continuous hold time in microseconds.
- wait_max
- The longest continuous wait time in microseconds.
- total
- The total (accumulated) hold time in microseconds.
- wait_total
- The total (accumulated) wait time in microseconds.
- count
- The total number of acquisitions.
- avg
- The average hold time in microseconds, derived from the
total hold time and the number of acquisitions.
- wait_avg
- The average wait time in microseconds, derived from the
total wait time and the number of acquisitions.
- cnt_hold
- The number of times the lock was held and another
thread attempted to acquire the lock.
- cnt_lock
- The number of times the lock was already held when this
point was reached.
- name
- The name of the acquisition point, derived from the
source file name and line number, followed by the name of the lock in
parentheses.
- debug.lock.prof.rejected
- The number of acquisition points that were ignored after
the table filled up.
- debug.lock.prof.skipspin
- Disable or enable the lock profiling code for the spin
locks. This defaults to 0 (do profiling for the spin locks).
- debug.lock.prof.skipcount
- Do sampling approximately every N lock acquisitions.
sysctl(8),
mutex(9)
Mutex profiling support appeared in
FreeBSD 5.0.
Generalized lock profiling support appeared in
FreeBSD
7.0.
The
MUTEX_PROFILING code was written by
Eivind Eklund
<
[email protected]>,
Dag-Erling Smørgrav
<
[email protected]>
and
Robert Watson
<
[email protected]>.
The
LOCK_PROFILING code was written by
Kip Macy
<
[email protected]>.
This manual page was written by
Dag-Erling
Smørgrav
<
[email protected]>.
The
LOCK_PROFILING option increases the size
of
struct lock_object, so a kernel built with
that option will not work with modules built without it.
The
LOCK_PROFILING option also prevents
inlining of the mutex code, which can result in a fairly severe performance
penalty. This is, however, not always the case.
LOCK_PROFILING can introduce a substantial
performance overhead that is easily monitorable using other profiling tools,
so combining profiling tools with
LOCK_PROFILING is not recommended.
Measurements are made and stored in nanoseconds using
nanotime(9), (on architectures without a
synchronized TSC) but are presented in microseconds. This should still be
sufficient for the locks one would be most interested in profiling (those that
are held long and/or acquired often).
LOCK_PROFILING should generally not be used
in combination with other debugging options, as the results may be strongly
affected by interactions between the features. In particular,
LOCK_PROFILING will report higher than
normal
uma(9) lock contention when run with
INVARIANTS due to extra locking that occurs
when
INVARIANTS is present; likewise, using
it in combination with
WITNESS will lead to
much higher lock hold times and contention in profiling output.