Collaborative reinforcement learning for a two-robot job transfer flow-shop scheduling problem

Abstract

A two-robot flow-shop scheduling problem with n identical jobs and m machines is defined and evaluated for four robot collaboration levels corresponding to different levels of information sharing, learning and assessment: Full – robots work together, performing self and joint learning sharing full information; Pull – one robot decides when and if to learn from the other robot; Push – one robot may force the second to learn from it and None – each robot learns independently with no information sharing. Robots operate on parallel tracks, transporting jobs between successive machines, returning empty to a machine to move another job. The objective is to obtain a robot schedule that minimises makespan (Cmax) for machines with varying processing times. A new reinforcement learning algorithm is developed, using dual Q-learning functions. A novel feature in the collaborative algorithm is the assignment of different reward functions to robots; minimising robot idle time and minimising job waiting time. Such delays increase makespan. Simulation analyses with fast, medium and slow speed robots indicated that Full collaboration with a fast–fast robot pair was best according to minimum average upper bound error. The new collaborative algorithm provides a tool for finding optimal and near-optimal solutions to difficult collaborative multi-robot scheduling problems.