Testing the Validity and Utility of a Career and Technical Education Programs of Study Evaluation Instrument.

The purpose of this study was to build on the research already completed by the Association for Career and Technical Education (ACTE) in the development of ACTE’s quality Program of Study Framework and accompanying evaluation instrument to examine the validity of the instrument’s results and its utility when used independently by local educators in a pilot test. The study used a two-phase mixed-methods design. In the first phase, 39 participating programs were evaluated using the instrument, consisting of 102 items organized into 12 elements. Representatives of participating programs, either teachers or administrators, then provided quantitative and qualitative feedback about the instrument, and 23 of the programs submitted Perkins program performance data. Results on the program evaluation instrument were correlated with the Perkins performance data to determine the criterion validity of the instrument. Correlations were significant for two of the performance measures, student technical skill attainment and completion, and positive but not statistically significant for student placement. Results related to the utility of the instrument, including for ease of use, practicality across programs, and usefulness of information, were all positive, and participants provided suggestions to further increase the possibility of wide scale future use. In Phase Two of the study, results of Phase One were used to inform in-depth interviews with a sample of five participants. These interviews provided further insight on the validity and utility questions guiding the study. The study is significant because it helps to more clearly define the characteristics of a high-quality CTE program and provide a standardized way to determine and improve the quality of CTE programs for millions of students around the country.