2-3-4 Combination for Parallel Compression on the K Computer

The development of supercomputers has successfully helped us to carry on complicated simulation with exploded size of dataset. For visualizing such kind of large-scale dataset, reducing the data size by using compression methods is one of the most useful approach. Moreover, parallelization of compression algorithm can greatly improve the efficiency and resolve the limitation of memory size. However, in parallel compression algorithm, interprocessor communication is indispensable, while it is also a bottleneck problem, especially for the general cases that the number of processors is not power-of-two. Parallel POD (proper orthogonal decomposition) compression algorithm is such an example, the number of time steps must be power-of-two for the binary swap scheme. A method that can fully resolve this problem with low computational cost will be very popular. In this paper, we proposed such an approach called 2-3-4 combination approach, which can be simply implemented and also reach high performance of parallel computing algorithms. Furthermore, our method can obtain the best balance among all parallel computing processors. This is achieved by transferring the non-power-of-two problem into power-of-two problem to fully use the best balance feature of binary swap method. We evaluate our approach through applying it to the parallel POD compression algorithm on the K computer.