The software testing challenges and methods

The Software Testing Process (STP) raised many challenging issues in past decades of software development practice, several of which remain open. The System/Software under test (SUT) continually increases complexity of applied technology, software application domain model and corresponding process knowledge and experience. Today's SUT have billions of possible inputs and outputs. How does one obtain adequate test coverage with reasonable or even optimal number of test events i.e. test cases? How does one measure test effectiveness, efficacy, benefits, risks (confidence) of project success, availability of resources, budget, time allocated to STP? How does one plan, estimate, predict, control, evaluate and choose best test scenario among hundreds of possible (considered, available, feasible) number of test events (test cases)? How does one judge, decide if satisfied/not satisfied program behavior, Pass/Fail result, Go/Ngo decision after test run i.e. does have Test Oracle? This paper describes the major issues that are encountered while developing framework of Integrated and Optimized Software Testing Process (IOSTP). IOSTP framework combines few engineering and scientific areas such as: Design of Experiments, Modeling & Simulation, integrated practical software measurement, Six Sigma strategy, Earned (Economic) Value Management (EVM) and Risk Management (RM) methodology through simulation-based software testing scenarios at various abstraction levels of the SUT to manage stable (predictable and controllable) software testing process at lowest risk, at an affordable price and time. In order to significantly improve software testing efficiency and effectiveness for the detection and removal of requirements and design defects in our framework of IOSTP, during 3 years of our IOSTP framework deployment to STP we calculated overall value returned on each dollar invested i.e. ROI of 100:1.