Examining approaches to measuring trade-offs in design artifacts

(Work-in-Progress) One of the key features of engineering that differentiates it from science is trade-off decisions which affect the overall quality of a design artifact. Making trade-offs is a complex cognitive process that involves weighing possible outcomes against their respective benefits and costs in areas such as aesthetics, cost, degree of safety, and various performance indicators. Making trade-off decisions is an effective design practice, and is a key dimension of successful performance. Designers and design researchers have methods to assess design artifact quality. While useful, these methods are not necessarily easy to use nor do they indicate design competency. Moreover, they are not grounded in a definition of engineering design. The objective of this study was to complete a comprehensive review of design literature to synthesize common tools used by designers and design students to assess design artifact quality as well as approaches used to assess design quality, where trade-offs are inherently integrated into this decision-making to compare design process to design outcome measures. Results and visuals compare and contrast how these tools and approaches assess design artifacts in terms of: (1) encompassing multiple complementary and competing dimensions, (2) consistent and systematic application, and (3) design competency level communication. This work shows the common pitfalls in common design artifacts assessment tools and methods with respect to understanding a key design behavior, making trade-offs, and establishes the need for a comprehensive way to assess student designer trade-off decisions in design that are easy to use and conceptually grounded in a definition of engineering design.