Hierarchical Co-Consistency Quantization and Information Refining Binary Network for Facial Expression Recognition in Human–Robot Interaction

Abstract

Facial expression recognition (FER) has become a trending research topic in human–robot interaction (HRI). However, the conventional CNN-based FER methods encounter challenges related to robustness and computational efficiency, limiting their applicability in HRI contexts. Although weight binarization has been proved to be effective in reducing the computational complexity, the loss of accuracy in the FER task is significant. In this article, a hierarchical co-consistency quantization (HCQ) and information refining binary network (IRBN) is proposed for FER during HRI. The IRBN incorporates one-shot aggregation (OSA) and convolution with edge difference mask to preserve low-level texture features, while a facial expression semantic information refiner is proposed for filtering irrelevant and ambiguous semantic information of high-level abstract features. HCQ optimizes the IRBN through progressive sign function and layer-by-layer feature loss derived from both full-precision and binary networks, maintaining the strong feature extraction capability of the binary network. The preliminary application experimental result demonstrates the feasibility of our method in HRI scenarios.