AmbieGen: A search-based framework for autonomous systems testing

摘要

Thorough testing of safety-critical autonomous systems, such as self-driving cars, autonomous robots, and drones, is essential for detecting potential failures before deployment. One crucial testing stage is model-in-the-loop testing, where the system model is evaluated by executing various scenarios in a simulator. However, the search space of possible parameters defining these test scenarios is vast, and simulating all combinations is computationally infeasible. To address this challenge, we introduce AmbieGen, a search-based test case generation framework for autonomous systems. AmbieGen uses evolutionary search to identify the most critical scenarios for a given system, and has a modular architecture that allows for the addition of new systems under test, algorithms, and search operators. Currently, AmbieGen supports test case generation for autonomous robots and autonomous car lane keeping assist systems. In this paper, we provide a high-level overview of the framework's architecture and demonstrate its practical use cases. • AmbieGen is an evolutionary algorithm based test scenario generation tool. • The search algorithm maximizes the difficulty of test scenarios as well as their diversity. • The tool is customizable and can be used to test different robotic systems. • Current tool version includes test scenario generation for autonomous vehicles and mobile robots. • The tool can be accessed at: https://github.com/swat-lab-optimization/AmbieGen-tool .