Adaptive Loss Balancing for Multitask Learning of Object Instance Recognition and 3D Pose Estimation

Abstract

Object instance recognition and 3D pose estimation are important elements in robot vision technology. State-of-the-art methods improve the accuracy of both instance recognition and pose estimation using multitask learning. These methods use unified balancing parameters to integrate the loss of each task, which means task difficulties are the same for all objects. However, the method we propose can adjust the balancing parameters for each object. This idea is based on the assumption that task difficulties are different for each object, since the distinctiveness of object instances and poses depends on their appearance and shape. Our method sequentially estimates task difficulties for CNN based on the amount of loss change and calculates balancing parameters for each object. Our experiments show that our method improves the accuracy of both object instance recognition and pose estimation compared with state-of-the-art methods using the common LineMOD dataset.