Large-Scale Evaluation of an Active Safety Algorithm with EuroNCAP and US NCAP Scenarios in a Virtual Test Environment -- An Industrial Case Study

Context: Recently, test protocols from organizations like European New Car Assessment Programme (EuroNCAP) were extended to also cover active safety systems. Objective: The official EuroNCAP test protocol for Autonomous Emergency Braking (AEB)/Forward Collision Warning (FCW) systems explicitly defines to what extent a Vehicle-Under-Test (VUT) is allowed to vary in its lateral position. In addition, the United States New Car Assessment Programme (US NCAP) test protocol has broader tolerance ranges. The goal for automotive OEMs is to understand the impact of such allowed variations on a the overall vehicle's performance. Method: A simulation-based approach is outlined that allows systematic, large-scale analysis of such influences to effectively plan time-consuming and resource-intense real-world vehicle tests. Our models allow a profound analysis of an AEB algorithm by modeling and conducting more than 3,000 simulation runs with EuroNCAP's dynamic CCRm and CCRb scenarios including those with adopted USNCAP parameters. Results: Our structured analysis of such test procedures involving dynamic actors is the first of its kind in a relevant industrial setting. Several anomalies were unveiled under US NCAP conditions to support real-world test runs. Hence, we could show that the proposed method supports all possible scenarios in AEB consumer tests and scales as we had to timely process approx. 7.7GB of simulation data. Conclusion: To achieve the expected performance and to study a system's behavior in potential misuse cases from a functional point of view, large scale, model-based simulations complement traditional testing on proving ground.