Performance testing and performance improvement methods for communicating systems

In this thesis, I propose methods for solving yet unsolved problems of black-box performance testing of communicating systems. The main motivation of creating the presented methods was the fact that while black-box conformance testing has evolved methods for test generation and execution, black-box performance testing lacks such methods, and performance tests are designed in an ad-hoc way. The thesis has three main parts. The first two parts focus on two different aspects of performance testing, while the third part discusses a problem, which is based on the second part. In the first part of the thesis, I deal with a problem of performance test execution namely, how to assign load generating software entities to the hosts of the performance test environment in order to exploit the capacities of the test environment as much as possible. After proving the NP-completeness of the problem, I formulate it as an integer linear program, and propose two heuristic methods for solving it. Then, the efficiency of the proposed methods are evaluated. In the second part, I focus on a more specific case of performance testing in which, the performance requirement that the system under test (SUT) has to fulfill is the number of request messages to be processed within a second while serving a given number of users. I propose a performance testing method, which automatically checks whether the SUT fulfills this performance requirement. The presented method builds on the functional modeling techniques used in conformance testing. I compare the accuracy of the proposed method to that of an ad-hoc performance testing method used in the industry. The third part of the thesis deals with the problem of increasing the performance of the SUT in case it has failed to pass the performance testing method presented in the second part of the thesis. After proving the NP-completeness of the problem, I formulate it as an integer linear program, and propose a heuristic algorithm for solving it. I also evaluate the efficiency of the two methods.