A parallelism-based analytic approach to performance evaluation using application programs

A brief overview of performance evaluation and benchmarking is presented. It is shown that traditional performance measurements recorded in these activities are really a direct measurement of the parallelism in software and hardware. A framework called the 'Path to Performance' that identifies the agents and activities that change the parallelism as it moves from problem to solution is developed. It is shown where the various current application benchmarks apply probes on the Path and which agents are being measured. The traditional time-based performance measurements are recast into parallelism-based performance measurements to show that understanding performance implies understanding the parallelism. A simulation tool and methodology are described for measuring and comparing the executed parallelism on a single CRAY Y-MP CPU. Application of the methodology to several of the Perfect Benchmarks suggests that the benchmarking utility of these programs on the Y-MP may be questioned because they exercise the machine in the same way.>