Testing Environments for Assessing Conformance and Interoperability.

We present a classification framework of testing environments that supports conformance and interoperability testing of distributed systems. We describe each testing environment and state the applicability and requirements placed on a realization of a testing tool. The broad categories identified include data instance validation, isolated systems testing, and peer-to-peer systems testing. Our targeted systems include those that support data exchange standards. We list and define the types of conformance and interoperability testing that can be conducted and associate the types that can be performed in a given testing environment. We describe and illustrate a conceptual test tool design for each testing environment. The delineation of environments and their testing capacity is intended to facilitate a more structured approach to testing in which the relationship between testing and test requirements is more clearly defined and the capabilities and limitations of testing tools are better understood. In broader terms, this approach should help industry and testing bodies identify and describe the depth and scope of their testing endeavors.