Situation coverage – a coverage criterion for testing autonomous robots

Abstract

Autonomous robots (AR) can get themselves into a wide range of situations, and they do not have a human to look after them in fine detail all the time. When we test autonomous robots, we must therefore care deeply about the range and diversity of the situations in which we have simulated and tested them – we must make sure that the situation coverage of our testing is adequate. Situation coverage measures can be implemented quantitatively, and so unlock a range of automated testing strategies. There are epistemic challenges to justifying the confidence we should attach to test results driven by situation coverage, but they are not fundamentally more difficult than those faced by other coverage criteria.