Evaluating the Memory Performance of a ccNUMA System

Scalable cache-coherent nonuniform memory access (ccNUMA) architectures are an important design segment for high-performance scalable multiprocessor systems. In order to write application programs that take advantage of such systems, or port application programs written for symmetric multiprocessor systems with uniform memory access times, it is important to understand the impact of nonuniform memory access times and the associated ccNUMA cache coherence protocols on aggregate application memory performance. This work presents a detailed memory performance analysis of a particular ccNUMA system (the SGI Origin 2000). The thesis presents a new memory profiling tool, called snperf, and a new set of microbenchmark codes, called snbench, which make such a fine-grained memory performance analysis possible. The analysis was performed on a wide variety of Origin 2000 system configurations and demonstrates that memory locality has a strong impact on application performance. More importantly, the results demonstrate a variety of second-order memory performance effects that are also substantial performance influences. Even though the specific implementation target for this thesis was the Origin 2000 architecture, the methods are applicable to other ccNUMA systems.