How to Benchmark Supercomputers

Benchmarks are designed to mimic a particular type of workload on a component or system of supercomputer to compare platforms, identify performance bottlenecks, evaluate potential solutions, and help users in their decision of buying or using machines most appropriate for their application requirements. In this paper we discuss the purpose, significance and method of benchmarking supercomputers, describe the state of the art, and review five of the mainstream benchmarks for supercomputer evaluation, among them, Linpack is the most popular for the famous TOP500 ranking list. In addition to this, there is the HPCG benchmark the goals of which are to stress a balance of floating point operation speed and communication bandwidth and latency, there is the Rodinia benchmark for evaluating the heterogeneous supercomputers composed of both GPUs and multi-core CPUs, there is the SPEC benchmark which can be applied to the newest generation of high-performance computers, there is still the DEISA benchmark comprises a number of real applications taken from a wide range of disciplines, including astrophysics, fluid dynamics, climate modelling, biosciences, materials science, fusion power and fundamental particle physics.