Enhancing DataRaceBench for Evaluating Data Race Detection Tools
Event Type
Workshop
Correctness
OpenMP
W
TimeWednesday, 11 November 20204:50pm - 5:15pm EDT
LocationTrack 10
DescriptionDataRaceBench is a dedicated benchmark suite to evaluate tools aimed to find data race bugs in OpenMP programs. Since its initial release in 2017, DataRaceBench has been widely used by tool developers to find the strengths and limitations of their tools. The results also provide an apple-to-apple comparison of the state-of-the-art of data race detection tools. In this paper, we discuss our latest efforts to enhance DataRaceBench. In particular, we have added support for the Fortran language and some of the newest OpenMP 5.0 language features. We also added new kernels representing new patterns from literature and other benchmarks (e.g., NAS Parallel Benchmark). In order to reduce duplicated code patterns in the benchmark suite, we have designed a distance-based code similarity analysis, combining both static and dynamic code features. Finally, we dockerize tools and streamline the entire benchmarking process to quickly generate a dashboard showing the state-of-the-art of data race detection of OpenMP programs. The enhanced DataRaceBench is released as v 1.3.0, with 222 newly added benchmarks. 56 of them are in C and the remaining 166 are in Fortran reproducing the C programs' nature. Our experiments show that this new version can spot more limitations of the current data race detection tools, with significantly reduced human intervention.
Author/Presenters
