Multi-Modal Human-Aware Image Caption System for Intelligent Service Robotics Applications

摘要

Image captioning is a high-level task that generates the context scenario descriptions from an image. There are many ways to implement such kinds of ability that can generate a smooth sentence about the context scenario. Because of the open and extensive domain of the dataset, seldom of them can really help people to get the meaningful information of a specified scenario. In this paper, we propose a novel framework called Human-Aware Context Generator (HACG). We adopt the concept of dividing and conquer, which combines the ability of face recognition and facial expression recognition while retaining the capability of image caption model. This model aims to offer a significant context scenario sentence to the people, especially for those who are urgent to master the overall situation.