Visual Novelty Detection for Mobile Inspection Robots

摘要

To achieve novelty detection on visual images, for the purpose of automated inspection tasks, we used a mechanism of visual attention that selects candidate image patches in the input frame, combined with methods that classify image patches as novel or non-novel. For real world inspection applications vision is the most appropriate sensor modality, as it provides colour, texture, shape, size, and distance information. All of this information is useful for robots operating in complex environments, but of course comes at the cost of processing large amounts of data, which is a particular challenge on autonomous mobile robots with limited computing power available. We demonstrated that the use of the saliency map (Itti et al., 1998) as selective attention mechanism minimises the amount of data to be processed and at the same time makes it possible to localise where the novel features are in the image frame. The use of this attention mechanism avoids explicit segmentation of the input image (Singh & Markou, 2004) and makes the overall system more robust to translations due to robot motion. Because novelty is of contextual nature and therefore can not be easily modelled, the approach that we follow is to first acquire a model of normality through robot learning and then use it as a means to highlight any abnormal features that are introduced in the environment. For this purpose, we have used unsupervised clustering mechanisms such as the GWR neural network (Marsland et al., 2002b) and the incremental PCA algorithm (Artac et al., 2002), which were both able to learn aspects of the environment incrementally and yielded very good results. We proposed an experimental setup to evaluate performance and functionality of visual novelty filters, dividing the experimental procedure in two stages: an exploration phase, in which the learning mechanism was enabled to allow the robot to build a model of normality while experiencing the environment; and an inspection phase, in which the acquired model of normality was used as a novelty filter. Novel objects were inserted in the robot's environment during the inspection phase of experiments with the expected outcome that the visual novelty filter would produce indications of novelty and localise these new objects in the corresponding input image frame. As the precise location and nature of the novelty introduced during the inspection phase is known by the experimenter, it is possible to generate ground truth data to be compared with the responses given by the novelty filter. In order to assess the performance of a novelty filter objectively, we used 2?2 contingency tables relating actual novelty status (ground truth) to system response, followed by the computation of statistical tests to quantify the association or agreement between them. Here we used the 2 test in order to check the statistical significance of the association between ground truth and novelty filter response, followed by the computation of Cramer's V, the uncertainty coefficient U and the index of agreement (Sachs, 2004). Extensive experimental data was logged to evaluate and compare the efficiency of the visual novelty filter. The 2 analysis of the generated contingency tables revealed statistical significance in the associations between system response and actual novelty status in all of the reported experiments. Typical quantitative analyses resulted in strong agreement with the ground truth data. Qualitative assessment of the learning procedure during exploration, as well as consistent identification of novel features during inspection was made through the use of novelty bar