Detecting Vulnerable Java Classes Based on the Analysis of Java Library Call Graph

Java Virtual Machine relies on the SecurityManager class to prevent illegal system resource access by external Java code (e.g., Java Applet). The SecurityManager checks the access right of all Java classes in the call stack when a system resource is requested and rejects the request if any of these classes does not have the proper access right. But researchers have found a number of vulnerable Java system classes that allow user programs to bypass the SecurityManager. The identification of these vulnerable system classes is important in order to strengthen the security of Java Virtual Machine. However, finding them manually in the huge space of Java libraries is not trivial task. In this paper, we suggest a systematic technique of identifying vulnerable Java classes in a given Java library through the analysis of the call graph. We observe that there is a number of Java methods that force the SecurityManager omit the full examination of the Java call stack for performance issue and that these target methods are exploited through the vulnerable Java classes to enable the bypassing of the SecurityManager. Based on this observation, our technique enumerates all Java classes that call these target methods, analyzes the call chains and successfully detects vulnerable classes.