Affinity-aware synchronization in work stealing run-times for NUMA multi-core processors

Abstract

Modern high-performance server systems are typically built as several multi-core chips put together in a single system. Each chip is connected to its local memory via an integrated memory controller (IMC) behaving as a node and hence the single machine behaving as non-uniform memory architecture (NUMA). Various user-level run-time systems adapt work stealing load balancing technique in multi-core processors. The work stealing run-times have to be aware of the topology of the processor on which they are running. Work stealing run-times on multi-core processors typically rely on lock-based synchronization to guarantee the coherency of shared mutable state. Synchronization constructs such as mutex locks, condition variables, and barriers are extensively used in implementation of these user-level work stealing run-times. The locality of these lock variables in multi-socket NUMA processors has considerable impact on the performance of these run-time systems. This paper studies the effect of locality of these synchronization constructs and proposes NUMA awareness to them. The proposed methodology is implemented using a source to source translator of OpenMP run-time, evaluated using OpenMP microbenchmark programs.