Developing a Standardized Performance Evaluation of Vehicles with Automated Driving Features

Objectives: The project goal was to create an initial set of standardized tests to explore whether they enable the ongoing evaluation of automated driving features as they evolve over time. These tests focused on situations that were representative of several daily driving scenarios as encountered by lower-level automated features, often called Advanced Driver Assistance Systems (ADAS), while looking forward to higher levels of automation as new systems are deployed. Methods: The research project initially gathered information through a review of existing literature about ADAS and current test procedures. Thereafter, a focus group of industry experts was convened for additional insights and feedback. With this background, the research team developed a series of tests designed to evaluate a variety of automated driving features in currently available implementations and anticipated future variants. Key ADAS available on current production vehicles include adaptive cruise control (ACC), lane keeping assist (LKA), and automatic emergency braking (AEB). Seven of the most automated production vehicles available in 2018 from six manufacturers were subjected to a series of standardized tests that were performed on a closed test track environment to assess the vehicle capabilities and limitations of the automated driving systems’ operational domains. Results: Considerable performance variability was observed between different vehicle manufacturers and within a single vehicle model across repeated trials and multiple replications. In addition, there were specific roadway characteristics that significantly impacted performance. Conclusions: The results indicate that standardized testing can assist researchers in determining the current capabilities of vehicles with automated driving features. The research team suggests continuing to improve and expand standardized testing of automated driving features and to work toward industry consensus of a robust evaluation mechanism that may play a key role in the conformance of future automated-vehicle systems.