http://valgrind.org/docs/manual/drd-manual.html
8. DRD: a thread error detector
To use this tool, you must specify
--tool=drd
on the Valgrind command line.
DRD is a Valgrind tool for detecting errors in multithreaded C and C++
shared-memory programs. The tool works for any program that uses the
POSIX threading primitives or that uses threading concepts built on
top of the POSIX threading primitives.
8.1.1. Multithreaded
Programming Paradigms
For many applications multithreading is a necessity. There are two
reasons why the use of threads may be required:
-
To model concurrent activities. Managing the state of one
activity per thread can be a great simplification compared to
multiplexing the states of multiple activities in a single thread. This
is why most server and embedded software is multithreaded.
-
To let computations run on multiple CPU cores simultaneously.
This is why many High Performance Computing (HPC) applications are
multithreaded.
Multithreaded programs can use one or more of the following
paradigms. Which paradigm is appropriate a.o. depends on the
application type -- modeling concurrent activities versus HPC.
Some examples of multithreaded programming paradigms are:
-
Locking. Data that is shared between threads may only be
accessed after a lock has been obtained on the mutex associated with
the shared data item. A.o. the POSIX threads library, the Qt library
and the Boost.Thread library support this paradigm directly.
-
Message passing. No data is shared between threads, but threads
exchange data by passing messages to each other. Well known
implementations of the message passing paradigm are MPI and CORBA.
-
Automatic parallelization. A compiler converts a sequential
program into a multithreaded program. The original program may or may
not contain parallelization hints. As an example, gcc supports the OpenMP standard from
gcc version 4.3.0 on. OpenMP is a set of compiler directives which tell
a compiler how to parallelize a C, C++ or Fortran program.
-
Software Transactional Memory (STM). Data is shared between
threads, and shared data is updated via transactions. After each
transaction it is verified whether there were conflicting transactions.
If there were conflicts, the transaction is aborted, otherwise it is
committed. This is a so-called optimistic approach. There is a
prototype of the Intel C Compiler (icc)
available that supports STM. Research is ongoing about the addition of
STM support to gcc.
DRD supports any combination of multithreaded programming paradigms as
long as the implementation of these paradigms is based on the POSIX
threads primitives. DRD however does not support programs that use
e.g. Linux' futexes directly. Attempts to analyze such programs with
DRD will cause DRD to report many false positives.
8.1.2. POSIX
Threads Programming Model
POSIX threads, also known as Pthreads, is the most widely available
threading library on Unix systems.
The POSIX threads programming model is based on the following
abstractions:
-
A shared address space. All threads running within the same
process share the same address space. All data, whether shared or not,
is identified by its address.
-
Regular load and store operations, which allow to read values
from or to write values to the memory shared by all threads running in
the same process.
-
Atomic store and load-modify-store operations. While these are
not mentioned in the POSIX threads standard, most microprocessors
support atomic memory operations. And some compilers provide direct
support for atomic memory operations through built-in functions like
e.g. __sync_fetch_and_add() which
is supported by both gcc and icc.
-
Threads. Each thread represents a concurrent activity.
-
Synchronization objects and operations on these synchronization
objects. The following types of synchronization objects are defined in
the POSIX threads standard: mutexes, condition variables, semaphores,
reader-writer locks, barriers and spinlocks.
Which source code statements generate which memory accesses depends on
the memory model of the
programming language
being used. There is not yet a definitive memory model for the C and
C++ languagues. For a draft memory model, see also document
WG21/N2338.
For more information about POSIX threads, see also the Single UNIX
Specification version 3, also known as
IEEE
Std 1003.1.
8.1.3. Multithreaded
Programming Problems
Depending on which multithreading paradigm is being used in a program,
one or more of the following problems can occur:
-
Data races. One or more threads access the same memory location
without sufficient locking.
-
Lock contention. One thread blocks the progress of one or more
other threads by holding a lock too long.
-
Improper use of the POSIX threads API. The most popular POSIX
threads implementation, NPTL, is optimized for speed. The NPTL will not
complain on certain errors, e.g. when a mutex is locked in one thread
and unlocked in another thread.
-
Deadlock. A deadlock occurs when two or more threads wait for
each other indefinitely.
-
False sharing. If threads that run on different processor cores
access different variables located in the same cache line frequently,
this will slow down the involved threads a lot due to frequent exchange
of cache lines.
Although the likelihood of the occurrence of data races can be reduced
through a disciplined programming style, a tool for automatic
detection of data races is a necessity when developing multithreaded
software. DRD can detect these, as well as lock contention and
improper use of the POSIX threads API.
8.1.4. Data
Race Detection
Synchronization operations impose an order on interthread memory
accesses. This order is also known as the happens-before relationship.
A multithreaded program is data-race free if all interthread memory
accesses are ordered by synchronization operations.
A well known way to ensure that a multithreaded program is data-race
free is to ensure that a locking discipline is followed. It is e.g.
possible to associate a mutex with each shared data item, and to hold
a lock on the associated mutex while the shared data is accessed.
All programs that follow a locking discipline are data-race free, but
not all data-race free programs follow a locking discipline. There
exist multithreaded programs where access to shared data is arbitrated
via condition variables, semaphores or barriers. As an example, a
certain class of HPC applications consists of a sequence of
computation steps separated in time by barriers, and where these
barriers are the only means of synchronization.
There exist two different algorithms for verifying the correctness of
multithreaded programs at runtime. The so-called Eraser algorithm
verifies whether all shared memory accesses follow a consistent
locking strategy. And the happens-before data race detectors verify
directly whether all interthread memory accesses are ordered by
synchronization operations. While the happens-before data race
detection algorithm is more complex to implement, and while it is more
sensitive to OS scheduling, it is a general approach that works for
all classes of multithreaded programs. Furthermore, the happens-before
data race detection algorithm does not report any false positives.
DRD is based on the happens-before algorithm.
8.2.1. Command Line
Options
The following command-line options are available for controlling the
behavior of the DRD tool itself:
-
--check-stack-var=<yes|no>
[default: no]
-
Controls whether DRD reports
data races for stack variables. This is disabled by default in order to
accelerate data race detection. Most programs do not share stack
variables over threads.
-
--exclusive-threshold=<n>
[default: off]
-
Print an error message if any mutex or writer lock has been
held longer than the specified time (in milliseconds). This option
enables detecting lock contention.
-
--report-signal-unlocked=<yes|no> [default: yes]
-
Whether to report calls to pthread_cond_signal()
and pthread_cond_broadcast() where the
mutex associated with the signal through pthread_cond_wait()
or pthread_cond_timed_wait()is not
locked at the time the signal is sent. Sending a signal without holding
a lock on the associated mutex is a common programming error which can
cause subtle race conditions and unpredictable behavior. There exist
some uncommon synchronization patterns however where it is safe to send
a signal without holding a lock on the associated mutex.
-
--segment-merging=<yes|no>
[default: yes]
-
Controls segment merging. Segment merging is an algorithm to
limit memory usage of the data race detection algorithm. Disabling
segment merging may improve the accuracy of the so-called 'other
segments' displayed in race reports but can also trigger an out of
memory error.
-
--shared-threshold=<n>
[default: off]
-
Print an error message if a reader lock has been held longer
than the specified time (in milliseconds). This option enables
detection of lock contention.
-
--show-confl-seg=<yes|no>
[default: yes]
-
Show conflicting segments in race reports. Since this
information can help to find the cause of a data race, this option is
enabled by default. Disabling this option makes the output of DRD more
compact.
-
--show-stack-usage=<yes|no>
[default: no]
-
Print stack usage at thread exit time. When a program creates a
large number of threads it becomes important to limit the amount of
virtual memory allocated for thread stacks. This option makes it
possible to observe how much stack memory has been used by each thread
of the the client program. Note: the DRD tool allocates some temporary
data on the client thread stack itself. The space necessary for this
temporary data must be allocated by the client program, but is not
included in the reported stack usage.
-
--var-info=<yes|no>
[default: no]
-
Display the names of global, static and stack variables when a
data race is reported. While this information can be very helpful, it
is not loaded into memory by default. This is because for big programs
reading in all debug information at once may cause an out of memory
error.
The following options are available for monitoring the behavior of the
client program:
-
--trace-addr=<address>
[default: none]
-
Trace all load and store activity for the specified address.
This option may be specified more than once.
-
--trace-barrier=<yes|no>
[default: no]
-
Trace all barrier activity.
-
--trace-cond=<yes|no>
[default: no]
-
Trace all condition variable activity.
-
--trace-fork-join=<yes|no>
[default: no]
-
Trace all thread creation and all thread termination events.
-
--trace-mutex=<yes|no>
[default: no]
-
Trace all mutex activity.
-
--trace-rwlock=<yes|no>
[default: no]
-
Trace all reader-writer lock activity.
-
--trace-semaphore=<yes|no>
[default: no]
-
Trace all semaphore activity.
8.2.2. Detected
Errors: Data Races
DRD prints a message every time it detects a data race. Please keep
the following in mind when interpreting DRD's output:
-
Every thread is assigned two thread
ID's: one thread ID is assigned by the Valgrind core and
one thread ID is assigned by DRD. Both thread ID's start at one.
Valgrind thread ID's are reused when one thread finishes and another
thread is created. DRD does not reuse thread ID's. Thread ID's are
displayed e.g. as follows: 2/3, where the first number is Valgrind's
thread ID and the second number is the thread ID assigned by DRD.
-
The term segment refers
to a consecutive sequence of load, store and synchronization
operations, all issued by the same thread. A segment always starts and
ends at a synchronization operation. Data race analysis is performed
between segments instead of between individual load and store
operations because of performance reasons.
-
There are always at least two memory accesses involved in a
data race. Memory accesses involved in a data race are called conflicting memory accesses. DRD
prints a report for each memory access that conflicts with a past
memory access.
Below you can find an example of a message printed by DRD when it
detects a data race:
$ valgrind --tool=drd --var-info=yes drd/tests/rwlock_race
...
==9466== Thread 3:
==9466== Conflicting load by thread 3/3 at 0x006020b8 size 4
==9466== at 0x400B6C: thread_func (rwlock_race.c:29)
==9466== by 0x4C291DF: vg_thread_wrapper (drd_pthread_intercepts.c:186)
==9466== by 0x4E3403F: start_thread (in /lib64/libpthread-2.8.so)
==9466== by 0x53250CC: clone (in /lib64/libc-2.8.so)
==9466== Location 0x6020b8 is 0 bytes inside local var "s_racy"
==9466== declared at rwlock_race.c:18, in frame #0 of thread 3
==9466== Other segment start (thread 2/2)
==9466== at 0x4C2847D: pthread_rwlock_rdlock* (drd_pthread_intercepts.c:813)
==9466== by 0x400B6B: thread_func (rwlock_race.c:28)
==9466== by 0x4C291DF: vg_thread_wrapper (drd_pthread_intercepts.c:186)
==9466== by 0x4E3403F: start_thread (in /lib64/libpthread-2.8.so)
==9466== by 0x53250CC: clone (in /lib64/libc-2.8.so)
==9466== Other segment end (thread 2/2)
==9466== at 0x4C28B54: pthread_rwlock_unlock* (drd_pthread_intercepts.c:912)
==9466== by 0x400B84: thread_func (rwlock_race.c:30)
==9466== by 0x4C291DF: vg_thread_wrapper (drd_pthread_intercepts.c:186)
==9466== by 0x4E3403F: start_thread (in /lib64/libpthread-2.8.so)
==9466== by 0x53250CC: clone (in /lib64/libc-2.8.so)
...
The above report has the following meaning:
-
The number in the column on the left is the process ID of the
process being analyzed by DRD.
-
The first line ("Thread 3") tells you Valgrind's thread ID for
the thread in which context the data race was detected.
-
The next line tells which kind of operation was performed (load
or store) and by which thread. Both Valgrind's and DRD's thread ID's
are displayed. On the same line the start address and the number of
bytes involved in the conflicting access are also displayed.
-
Next, the call stack of the conflicting access is displayed. If
your program has been compiled with debug information (-g), this call
stack will include file names and line numbers. The two bottommost
frames in this call stack (clone and start_thread) show how the NPTL starts a
thread. The third frame (vg_thread_wrapper)
is added by DRD. The fourth frame (thread_func)
is the first interesting line because it shows the thread entry point,
that is the function that has been passed as the third argument to pthread_create().
-
Next, the allocation context for the conflicting address is
displayed. For dynamically allocated data the allocation call stack is
shown. For static variables and stack variables the allocation context
is only shown when the option --var-info=yes
has been specified. Otherwise DRD will print Allocation context: unknown.
-
A conflicting access involves at least two memory accesses. For
one of these accesses an exact call stack is displayed, and for the
other accesses an approximate call stack is displayed, namely the start
and the end of the segments of the other accesses. This information can
be interpreted as follows:
-
Start at the bottom of both call stacks, and count the
number stack frames with identical function name, file name and line
number. In the above example the three bottommost frames are identical (clone, start_thread
and vg_thread_wrapper).
-
The next higher stack frame in both call stacks now tells
you between in which source code region the other memory access
happened. The above output tells that the other memory access involved
in the data race happened between source code lines 28 and 30 in file rwlock_race.c.
8.2.3. Detected
Errors: Lock Contention
Threads must be able to make progress without being blocked for too
long by other threads. Sometimes a thread has to wait until a mutex or
reader-writer lock is unlocked by another thread. This is called
lock contention.
Lock contention causes delays. Such delays should be as short as
possible. The two command line options
--exclusive-threshold=<n> and
--shared-threshold=<n> make it
possible to
detect excessive lock contention by making DRD report any lock that
has been held longer than the specified threshold. An example:
$ valgrind --tool=drd --exclusive-threshold=10 drd/tests/hold_lock -i 500
...
==10668== Acquired at:
==10668== at 0x4C267C8: pthread_mutex_lock (drd_pthread_intercepts.c:395)
==10668== by 0x400D92: main (hold_lock.c:51)
==10668== Lock on mutex 0x7fefffd50 was held during 503 ms (threshold: 10 ms).
==10668== at 0x4C26ADA: pthread_mutex_unlock (drd_pthread_intercepts.c:441)
==10668== by 0x400DB5: main (hold_lock.c:55)
...
The hold_lock test program holds a lock
as long as
specified by the -i (interval) argument.
The DRD
output reports that the lock acquired at line 51 in source file
hold_lock.c and released at line 55 was
held during
503 ms, while a threshold of 10 ms was specified to DRD.
8.2.4. Detected
Errors: Misuse of the POSIX threads API
DRD is able to detect and report the following misuses of the POSIX
threads API:
-
Passing the address of one type of synchronization object (e.g.
a mutex) to a POSIX API call that expects a pointer to another type of
synchronization object (e.g. a condition variable).
-
Attempts to unlock a mutex that has not been locked.
-
Attempts to unlock a mutex that was locked by another thread.
-
Attempts to lock a mutex of type PTHREAD_MUTEX_NORMAL
or a spinlock recursively.
-
Destruction or deallocation of a locked mutex.
-
Sending a signal to a condition variable while no lock is held
on the mutex associated with the signal.
-
Calling pthread_cond_wait() on a
mutex that is not locked, that is locked by another thread or that has
been locked recursively.
-
Associating two different mutexes with a condition variable
through pthread_cond_wait().
-
Destruction or deallocation of a condition variable that is
being waited upon.
-
Destruction or deallocation of a locked reader-writer lock.
-
Attempts to unlock a reader-writer lock that was not locked by
the calling thread.
-
Attempts to recursively lock a reader-writer lock exclusively.
-
Reinitialization of a mutex, condition variable, reader-writer
lock, semaphore or barrier.
-
Destruction or deallocation of a semaphore or barrier that is
being waited upon.
-
Exiting a thread without first unlocking the spinlocks, mutexes
or reader-writer locks that were locked by that thread.
Just as for other Valgrind tools it is possible to let a client
program interact with the DRD tool.
The interface between client programs and the DRD tool is defined in
the header file <valgrind/drd.h>.
The
available client requests are:
-
VG_USERREQ__DRD_GET_VALGRIND_THREAD_ID.
Query the thread ID that was assigned by the Valgrind core to the
thread executing this client request. Valgrind's thread ID's start at
one and are recycled in case a thread stops.
-
VG_USERREQ__DRD_GET_DRD_THREAD_ID.
Query the thread ID that was assigned by DRD to the thread executing
this client request. DRD's thread ID's start at one and are never
recycled.
-
VG_USERREQ__DRD_START_SUPPRESSION.
Some applications contain intentional races. There exist e.g.
applications where the same value is assigned to a shared variable from
two different threads. It may be more convenient to suppress such races
than to solve these. This client request allows to suppress such races.
See also the macro DRD_IGNORE_VAR(x)
defined in <valgrind/drd.h>.
-
VG_USERREQ__DRD_FINISH_SUPPRESSION.
Tell DRD to no longer ignore data races in the address range that was
suppressed via VG_USERREQ__DRD_START_SUPPRESSION.
-
VG_USERREQ__DRD_START_TRACE_ADDR.
Trace all load and store activity on the specified address range. When
DRD reports a data race on a specified variable, and it's not
immediately clear which source code statements triggered the
conflicting accesses, it can be helpful to trace all activity on the
offending memory location. See also the macro DRD_TRACE_VAR(x)
defined in <valgrind/drd.h>.
-
VG_USERREQ__DRD_STOP_TRACE_ADDR.
Do no longer trace load and store activity for the specified address
range.
Note: if you compiled Valgrind yourself, the header file
<valgrind/drd.h> will have been
installed in
the directory /usr/include by the command
make install. If you obtained Valgrind by
installing it as a package however, you will probably have to install
another package with a name like valgrind-devel
before Valgrind's header files are present.
8.2.6. Debugging GNOME
Programs
GNOME applications use the threading primitives provided by the
glib and
gthread libraries. These libraries
are built on top of POSIX threads, and hence are directly supported by
DRD. Please keep in mind that you have to call
g_thread_init() before creating any
threads, or
DRD will report several data races on glib functions. See also the
GLib
Reference Manual for more information about
g_thread_init().
One of the many facilities provided by the glib
library is a block allocator, called g_slice.
You
have to disable this block allocator when using DRD by adding the
following to the shell environment variables:
G_SLICE=always-malloc. See also the GLib
Reference Manual for more information.
8.2.7. Debugging Qt
Programs
The Qt library is the GUI library used by the KDE project. Currently
there are two versions of the Qt library in use: Qt3 by KDE 3 and Qt4
by KDE 4. If possible, use Qt4 instead of Qt3. Qt3 is no longer
supported, and there are known problems with multithreading support in
Qt3. As an example, using QString objects in more than one thread will
trigger race reports (this has been confirmed by Trolltech -- see also
Trolltech task #206152).
Qt4 applications are supported by DRD, but only if the
libqt4-debuginfo package has been
installed. Some
of the synchronization and threading primitives in Qt4 bypass the
POSIX threads library, and DRD can only intercept these if symbol
information for the Qt4 library is available. DRD won't tell you if it
has not been able to load the Qt4 debug information, but a huge number
of data races will be reported on data protected via
QMutex objects.
8.2.8. Debugging
Boost.Thread Programs
The Boost.Thread library is the threading library included with the
cross-platform Boost Libraries. This threading library is an early
implementation of the upcoming C++0x threading library.
Applications that use the Boost.Thread library should run fine under
DRD.
More information about Boost.Thread can be found here:
-
Anthony Williams, Boost.Thread Library Documentation, Boost website,
2007.
-
Anthony Williams, What's New in Boost Threads?, Recent changes to the
Boost Thread library, Dr. Dobbs Magazine, October 2008.
8.2.9. Debugging
OpenMP Programs
OpenMP stands for Open Multi-Processing.
The
OpenMP standard consists of a set of compiler directives for C, C++
and Fortran programs that allows a compiler to transform a sequential
program into a parallel program. OpenMP is well suited for HPC
applications and allows to work at a higher level compared to direct
use of the POSIX threads API. While OpenMP ensures that the POSIX API
is used correctly, OpenMP programs can still contain data races. So it
makes sense to verify OpenMP programs with a thread checking tool.
DRD supports OpenMP shared-memory programs generated by gcc. The gcc
compiler supports OpenMP since version 4.2.0. Gcc's runtime support
for OpenMP programs is provided by a library called
libgomp. The synchronization primites
implemented
in this library use Linux' futex system call directly, unless the
library has been configured with the
--disable-linux-futex flag. DRD only
supports
libgomp libraries that have been configured with this flag and in
which symbol information is present. For most Linux distributions this
means that you will have to recompile gcc. See also the script
drd/scripts/download-and-build-gcc in the
Valgrind source tree for an example of how to compile gcc. You will
also have to make sure that the newly compiled
libgomp.so library is loaded when OpenMP
programs
are started. This is possible by adding a line similar to the
following to your shell startup script:
export LD_LIBRARY_PATH=~/gcc-4.3.2/lib64:~/gcc-4.3.2/lib:
As an example, the test OpenMP test program
drd/tests/omp_matinv triggers a data race
when the option -r has been specified on the command line. The data
race is triggered by the following code:
#pragma omp parallel for private(j)
for (j = 0; j < rows; j++)
{
if (i != j)
{
const elem_t factor = a[j * cols + i];
for (k = 0; k < cols; k++)
{
a[j * cols + k] -= a[i * cols + k] * factor;
}
}
}
The above code is racy because the variable k
has
not been declared private. DRD will print the following error message
for the above code:
$ valgrind --check-stack-var=yes --var-info=yes --tool=drd drd/tests/omp_matinv 3 -t 2 -r
...
Conflicting store by thread 1/1 at 0x7fefffbc4 size 4
at 0x4014A0: gj.omp_fn.0 (omp_matinv.c:203)
by 0x401211: gj (omp_matinv.c:159)
by 0x40166A: invert_matrix (omp_matinv.c:238)
by 0x4019B4: main (omp_matinv.c:316)
Allocation context: unknown.
...
In the above output the function name gj.omp_fn.0
has been generated by gcc from the function name
gj. Unfortunately the variable name
k is not shown as the allocation context
-- it is
not clear to me whether this is caused by Valgrind or whether this is
caused by gcc. The most usable information in the above output is the
source file name and the line number where the data race has been
detected
(omp_matinv.c:203).
Note: DRD reports errors on the libgomp
library
included with gcc 4.2.0 up to and including 4.3.2. This might indicate
a race condition in the POSIX version of libgomp.
For more information about OpenMP, see also openmp.org.
8.2.10. DRD
and Custom Memory Allocators
DRD tracks all memory allocation events that happen via either the
standard memory allocation and deallocation functions
(malloc, free,
new and delete)
or via entry
and exit of stack frames. DRD uses memory allocation and deallocation
information for two purposes:
-
To know where the scope ends of POSIX objects that have not
been destroyed explicitly. It is e.g. not required by the POSIX threads
standard to call pthread_mutex_destroy()
before freeing the memory in which a mutex object resides.
-
To know where the scope of variables ends. If e.g. heap memory
has been used by one thread, that thread frees that memory, and another
thread allocates and starts using that memory, no data races must be
reported for that memory.
It is essential for correct operation of DRD that the tool knows about
memory allocation and deallocation events. DRD does not yet support
custom memory allocators, so you will have to make sure that any
program which runs under DRD uses the standard memory allocation
functions. As an example, the GNU libstdc++ library can be configured
to use standard memory allocation functions instead of memory pools by
setting the environment variable
GLIBCXX_FORCE_NEW. For more information,
see also
the libstdc++
manual.
8.2.11. DRD
Versus Memcheck
It is essential for correct operation of DRD that there are no memory
errors such as dangling pointers in the client program. Which means
that
it is a good idea to make sure that your program is memcheck-clean
before you analyze it with DRD. It is possible however that some of
the memcheck reports are caused by data races. In this case it makes
sense to run DRD before memcheck.
So which tool should be run first ? In case both DRD and memcheck
complain about a program, a possible approach is to run both tools
alternatingly and to fix as many errors as possible after each run of
each tool until none of the two tools prints any more error messages.
8.2.12. Resource
Requirements
The requirements of DRD with regard to heap and stack memory and the
effect on the execution time of client programs are as follows:
-
When running a program under DRD with default DRD options,
between 1.1 and 3.6 times more memory will be needed compared to a
native run of the client program. More memory will be needed if loading
debug information has been enabled (--var-info=yes).
-
DRD allocates some of its temporary data structures on the
stack of the client program threads. This amount of data is limited to
1 - 2 KB. Make sure that thread stacks are sufficiently large.
-
Most applications will run between 20 and 50 times slower under
DRD than a native single-threaded run. Applications such as Firefox
which perform very much mutex lock / unlock operations however will run
too slow to be usable under DRD. This issue will be addressed in a
future DRD version.
8.2.13. Hints
and Tips for Effective Use of DRD
The following information may be helpful when using DRD:
-
Make sure that debug information is present in the executable
being analysed, such that DRD can print function name and line number
information in stack traces. Most compilers can be told to include
debug information via compiler option -g.
-
Compile with flag -O1 instead of -O0. This will reduce the amount of generated
code, may reduce the amount of debug info and will speed up DRD's
processing of the client program. For more information, see also Getting
started.
-
If DRD reports any errors on libraries that are part of your
Linux distribution like e.g. libc.so or libstdc++.so, installing the debug packages for
these libraries will make the output of DRD a lot more detailed.
-
When using C++, do not send output from more than one thread to
std::cout. Doing so would not only
generate multiple data race reports, it could also result in output
from several threads getting mixed up. Either use printf() or do the following:
-
Derive a class from std::ostreambuf
and let that class send output line by line to stdout.
This will avoid that individual lines of text produced by different
threads get mixed up.
-
Create one instance of std::ostream
for each thread. This makes stream formatting settings thread-local.
Pass a per-thread instance of the class derived from std::ostreambuf to the constructor of each
instance.
-
Let each thread send its output to its own instance of std::ostream instead of std::cout.
8.3. Using
the POSIX Threads API Effectively
The Single UNIX Specification version two defines the following four
mutex types (see also the documentation of pthread_mutexattr_settype()):
-
normal, which means
that no error checking is performed, and that the mutex is
non-recursive.
-
error checking, which
means that the mutex is non-recursive and that error checking is
performed.
-
recursive, which means
that a mutex may be locked recursively.
-
default, which means
that error checking behavior is undefined, and that the behavior for
recursive locking is also undefined. Or: portable code must neither
trigger error conditions through the Pthreads API nor attempt to lock a
mutex of default type recursively.
In complex applications it is not always clear from beforehand which
mutex will be locked recursively and which mutex will not be locked
recursively. Attempts lock a non-recursive mutex recursively will
result in race conditions that are very hard to find without a thread
checking tool. So either use the error checking mutex type and
consistently check the return value of Pthread API mutex calls, or use
the recursive mutex type.
8.3.2. Condition
variables
A condition variable allows one thread to wake up one or more other
threads. Condition variables are often used to notify one or more
threads about state changes of shared data. Unfortunately it is very
easy to introduce race conditions by using condition variables as the
only means of state information propagation. A better approach is to
let threads poll for changes of a state variable that is protected by
a mutex, and to use condition variables only as a thread wakeup
mechanism. See also the source file
drd/tests/monitor_example.cpp for
an
example of how to implement this concept in C++. The monitor concept
used in this example is a well known and very useful concept -- see
also Wikipedia for more information about the monitor
concept.
8.3.3. pthread_cond_timedwait()
and timeouts
Historically the function
pthread_cond_timedwait() only allowed the
specification of an absolute timeout, that is a timeout independent of
the time when this function was called. However, almost every call to
this function expresses a relative timeout. This typically happens by
passing the sum of
clock_gettime(CLOCK_REALTIME) and a
relative timeout as the third argument. This approach is incorrect
since forward or backward clock adjustments by e.g. ntpd will affect
the timeout. A more reliable approach is as follows:
-
When initializing a condition variable through
pthread_cond_init(), specify that the timeout of
pthread_cond_timedwait() will use the clock CLOCK_MONOTONIC
instead of CLOCK_REALTIME. You can do
this via pthread_condattr_setclock(...,
CLOCK_MONOTONIC).
-
When calling pthread_cond_timedwait(),
pass the sum of clock_gettime(CLOCK_MONOTONIC)
and a relative timeout as the third argument.
See also
drd/tests/monitor_example.cpp for
an
example.
8.3.4. Assigning
names to threads
Many applications log information about changes in internal or
external state to a file. When analyzing log files of a multithreaded
application it can be very convenient to know which thread logged
which information. One possible approach is to identify threads in
logging output by including the result of
pthread_self() in every log line.
However, this approach
has two disadvantages: there is no direct relationship between these
values and the source code and these values can be different in each
run. A better approach is to assign a brief name to each thread and to
include the assigned thread name in each log line. One possible
approach for managing thread names is as follows:
-
Allocate a key for the pointer to the thread name through pthread_key_create().
-
Just after thread creation, set the thread name through pthread_setspecific().
-
In the code that generates the logging information, query the
thread name by calling pthread_getspecific().
DRD currently has the following limitations:
-
DRD has only been tested on the Linux operating system, and not
on any of the other operating systems supported by Valgrind.
-
Of the two POSIX threads implementations for Linux, only the
NPTL (Native POSIX Thread Library) is supported. The older LinuxThreads
library is not supported.
-
DRD, just like memcheck, will refuse to start on Linux
distributions where all symbol information has been removed from ld.so.
This is a.o. the case for the PPC editions of openSUSE and Gentoo. You
will have to install the glibc debuginfo package on these platforms
before you can use DRD. See also openSUSE bug
396197 and Gentoo bug 214065.
-
When DRD prints a report about a data race detected on a stack
variable in a parallel section of an OpenMP program, the report will
contain no information about the context of the data race location (Allocation context: unknown). It's not
yet clear whether this behavior is caused by Valgrind or by gcc.
-
When address tracing is enabled, no information on atomic
stores will be displayed. This functionality is easy to add however.
Please contact the Valgrind authors if you would like to see this
functionality enabled.
-
If you compile the DRD source code yourself, you need gcc 3.0
or later. Gcc 2.95 is not supported.
If you have any comments, suggestions, feedback or bug reports about
DRD, feel free to either post a message on the Valgrind users mailing
list or to file a bug report. See also http://www.valgrind.org/
for more information.
|
