Sketch Generation From Real Object Images Using Generative Adversarial Network and Deep Reinforcement Learning

Abstract

Technology in Robotics and machine learning have been applied in numerous fields including the arts. Paul The Robot is able to draw sketches from human faces using the conventional convolution filter method. Generative Adversarial Network (GAN) has been successful in generating synthetic images. Researches in sketch generation have been conducted either by using Recurrent Neural Network (RNN) or by using Deep Reinforcement Learning, with step-by-step stroke drawing. This research proposes a system to generate sketches from real object images using GAN dan Deep Reinforcement Learning. The training framework used is based on Doodle-SDQ (Doodle with Stroke Demonstration and Deep Q-Network) that combines supervised learning and reinforcement learning. Real object images are converted into contour images by GAN to be the reference images by the reinforcement learning agent to generate the sketch. The experiment is done by modifying pooling layers during the supervised learning stage and rare exploration scenarios during the reinforcement learning stage. The result of this research is a model that can reach an average total reward of 2558.98 with an average pixel error of 0.0489 using 200 as the maximum step in an average time of 3.29 seconds for the sketch generation.