Human Expertise Inspired Smart Sensing and Manufacturing

Abstract

With the new era of the fourth industrial revolution, adopting human skills and knowledge is getting more valuable to enable smart manufacturing. Moreover, a cognitive ability that comes from human sensory and intuition is able to facilitate smart sensing and collaboration between humans and autonomy. For smart manufacturing, the improvement of collaboration of humans, autonomy, and physical systems is essential. In this paper, an immersive and interactive cyber-physical system (I <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> CPS) is suggested to achieve efficient collaboration between humans and autonomy. Virtual reality (VR) with a game engine, a new middleware, and a data-to-information protocol were introduced for I <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> CPS. This system was utilized in an autonomous robot grasping platform constructed by human skills and self-learning. The initial convolutional neural network (CNN) model was trained by manual demonstration and showed 39% of the success rate of grasping blocks. After self-learning, the success rate increased to 70%. As another human-inspired sensing technique, sound recognition system using ambient sound and the CNN model to predict the running state of the generator is proposed. The Mel spectrum based on human hearings was used as feature extraction. The overall running prediction accuracy of the evaluation dataset showed approximately 95%. This system was applied for web-based remote and real-time monitoring of generators.