Design and evaluation of dynamic optimizations for a Java just-in-time compiler

The high performance implementation of Java Virtual Machines (JVM) and Just-In-Time (JIT) compilers is directed toward employing a dynamic compilation system on the basis of online runtime profile information. The trade-off between the compilation overhead and performance benefit is a crucial issue for such a system. This article describes the design and implementation of a dynamic optimization framework in a production-level Java JIT compiler, together with two techniques for profile-directed optimizations: method inlining and code specialization. Our approach is to employ a mixed mode interpreter and a three-level optimizing compiler, supporting level-1 to level-3 optimizations, each of which has a different set of trade-offs between compilation overhead and execution speed. A lightweight sampling profiler operates continuously during the entire period while applications are running to monitor the programs' hot spots. Detailed information on runtime behavior can be collected by dynamically generating instrumentation code that is installed to and uninstalled from the specified recompilation target code. Value profiling with this instrumentation mechanism allows fully automatic profile-directed method inlining and code specialization to be performed on the basis of call site information or specific parameter values at the higher optimization levels. The experimental results show that our approach offers high performance and low compilation overhead in both program startup and steady state measurements in comparison to the previous systems. The two profile-directed optimization techniques contribute significant portions of the improvements.