Empirical study of parallelism throttling schemes on a massively parallel system

Exploitation of parallelism in massively parallel systems is intuitively appealing and is a promising avenue for achieving teraflop performance. However, parallelism is not free; apart from overheads in communication and synchronisation, having too much program parallelism can also raise serious resource management problems during program execution. The problem of resource management is particularly complicated by the distributed nature of massively parallel systems. We address the issue of managing parallelism in a massively parallel system. This is an extension of our previous work on hardware throttle for parallel systems. We propose two parallelism throttle schemes, and a scheme for monitoring and measurement of system workload. Processes in the system are initiated when runtime loading levels in the system permit. The aim is to match dynamic program parallelism to static machine parallelism. An experimental study is conducted using a simulated multi-cluster dataflow machine as a testbed. Our study shows the extent that control over runaway program parallelism is necessary, and that it is possible to have good distributed control of resource use in a massively parallel system.>