Extreme programming evaluation framework for object-oriented languages -- version 1.2

The Extreme Programming (XP) software development methodology appears appropriate for some teams or projects.Anecdotal stories from developers who have adopted XP are often favorable.However, there is little empirical evidence to support or refute claims regarding XP.The Extreme Programming Evaluation Framework (XP-EF) provides a benchmark measurement framework for researchers and practitioners to assess concretely the extent to which an organization has adopted XP practices and the result of this adoption.Through a series of case studies that utilize the structure of the XP-EF, the community can build an experience factory of empirical findings on XP; the results of these case studies can be integrated and combined.This paper describes the elements of the XP-EF and provides examples of using it to measure a software project.• Concrete and unambiguous • Comprehensive and complete enough to cover vital factors Figure 1: Extreme Programming Evaluation FrameworkThis report gives instructions on how small informal teams can concretely and consistently measure XP using the XP-EF so that different teams can repeat similar studies.The XP-EF is a compilation of validated and proposed metrics.The validation of metric relevance requires convincing demonstration that (1) the metric measures what it purports to measure and (2) the metric is associated with an important external metric, such as field reliability, maintainability, or fault-proneness [9].Several of the XP-am metrics require validation via repeated use of the XP-EF, and will be expanded to provide a more comprehensive, objective assessment of adherence to XP practices.In empirical studies, comparisons are informative.For example, a new project's measures can be compared against a prior project's measures within the same organization.Alternatively, comparisons can be made to industry standards and/or benchmarks.Jones has compiled data from many software organizations and provides benchmarks, best practices, and statistics for a range of software development topics [12].Section 2 of this report explains the motivation and details behind the XP-Context Factors (XP-cf).Section 3 and 4 do the same for the XP-Adherence Metrics (XP-am) and XP-Outcome Measures (XP-om), respectively.A subject XP-Adherence survey, a customer satisfaction survey, and a programmer interview template are found in the appendix.Those who desire to see a completed XP-EF template and corresponding analysis are directed to the IBM case study [32] and the Sabre Airline Solutions case study [17].Examples from these case studies are provided throughout this paper. The XP-Context FactorsDrawing general conclusions from empirical studies in software engineering is difficult because the results of any process largely depend upon the specifics of the study and relevant context factors.We cannot assume a priori that a study's results generalize beyond the specific environment in which it was conducted [2].Therefore, recording an experiment's context factors is essential for comparison purposes and for fully understanding the similarities and differences between the case study and one's own environment.Under the guidelines of the GQM, we consider our goal for the defining the context factors: GOAL: To be able to understand in what ways a particular XP case study is similar or different from another XP case study or to understand in what ways a particular (XP) project is similar or different from a new (XP) project.QUESTION: In what ways can software projects differ?Extreme Programming Evaluation Framework XP-EF XP-Context Factors (XP-cf) XP-Adherence Metrics (XP-am) XP-Outcome Measures (XP-om)