1. High-Performance Computing

In this chapter we review some of the basic features of traditional and advanced computers. The review is not intended to be a complete discussion of the architecture of any particular machine or a detailed analysis of computer architectures. Rather, our focus is on certain features that are especially relevant to the implementation of linear algebra algorithms. 1.1 Trends in Computer Design In the past decade, the world has experienced one of the most exciting periods in computer development. Computer performance improvements have been dramatic—a trend that promises to continue for the next several years. One reason for the improved performance is the rapid advance in microprocessor technology. Microprocessors have become smaller, denser, and more powerful. Indeed, if cars had made equal progress, you could buy a car for a few dollars, drive it across the country in a few minutes, and “park” the car in your pocket! The result is that microprocessor-based supercomputing is rapidly becoming the technology of preference in attacking some of the most important problems of science and engineering. To exploit microprocessor technology, vendors have developed highly parallel computers. Highly parallel systems offer the enormous computational power needed for solving some of our most challenging computational problems such as simulating the climate. Unfortunately, software development has not kept pace with hardware advances. New programming paradigms, languages, scheduling and partitioning techniques, and algorithms are needed to fully exploit the power of these highly parallel machines. A major new trend for scientific problem solving is distributed computing. In distributed computing, computers connected by a network are used collectively to solve a single large problem.