Occluded Image Function: A Novel Measure for Evaluating Machine Learning Classifiers for Biometrics

Abstract

A novel method is presented for evaluating the efficacy of object recognition algorithms on occluded images, called the occluded image function (OIF). The OIF describes system behavior in occluded environments and thus gives qualitative insight into their mechanisms; derivative metrics from OIF can also be used to quantitatively compare classifiers. To showcase the utility of the OIF, an experiment is performed by obstructing optical gait images from two biped robot models and using four binary machine learning classifiers to distinguish between them. The OIF diagrams are created from each experiment, and the resulting insights about the classifiers are discussed. Using the OIF, it is shown that the primitive classifiers can sometimes perform better under occlusion conditions, possibly due to pre-filtering of gait data by uniform occlusions. This result serves to demonstrate that the OIF is a useful tool for classifier evaluation.