Generation of conformance test suites for parallel and distributed languages and APIs

This paper outlines a new methodology of systematic generation of conformance test suites. Conformance testing is aimed at verifying whether the implementation follows its specification, which is specialty important in parallel and distributed environments, where many different software packages must cooperate. The author started his research for conformance testing for a parallel data-driven language Athapascan, but first results are quite universal and the presented methodology can be used for the analysis of existing conformance test suites. An example of using this methodology to verify a part of POSIX Conformance Test Suite given by NIST presented in the paper illustrates advantages of the proposed methodology and reveals a serious error in the official conformance test suite widely used for about eight years.