A programmer performance measure based on programmer state transitions in testing and debugging process

To organize and manage software development teams, it as important to evaluate the capability of each programmer based on reliable and easily collected data. We present a system which automatically monitors programmer activities, and propose a programmer debugging performance measure based on data monitored by the system. The system automatically categorizes programmer activity in real time into three types (compilation, program execution, and program modification) by monitoring and analyzing key strokes of a programmer. The resulting outputs are the time sequences of monitored activities. The measure we propose is the average length of debugging time per fault, D, estimated from the data sequences monitored by the system. To estimate the debugging time per fault, we introduce a testing and debugging process model. The process model has parameters associated with the average length of a program modification, d, and the probability of a fault being fixed completely by a program modification, r. By taking account of r as well as d, the debugging time per fault can be estimated with high accuracy. The model parameters, such as d and r, are computed from the monitored data sequences by using a maximum likelihood estimation method.>