Layer-by-Layer Pick and Place Collaboration Between Human and Robot Using Optimization

Abstract

Robotic pick-and-place (P&P) has been widely utilized in manufacturing and architectural construction since the 1980s. However, the lack of inherent sensing capabilities in robots has limited their ability to adapt and respond to changes in design or environment. To address some of these shortcomings, this paper proposes an interactive robotic brick-laying workflow using a vision-based sensing framework to inform and optimize brick placements in consecutive layers. The proposed implementation is comprised of three major computational frameworks: (1) digitally reconstructing and analyzing the current state of the assembly, (2) optimizing placement targets based on the digital representation of the environment and desired multi-objective optimization goals, and (3) planning robot motion for the next layer of brick-laying. Within this workflow, the vision-based feedback pipeline simultaneously reconstructs and localizes the already-built assembly. This geometric information constitutes the basis for the multi-objective optimization stage. The placement targets are adaptively calculated to build the next layer upon the existing assembly while optimizing for structural stability, accounting for unforeseen deviations between layers, and allowing for human intervention and modification throughout the process. By proposing an interactive robotic brick-laying workflow, the paper explores the prospects for leveraging the capabilities of robotic pick-and-place technology and integrating it with vision-based sensing frameworks to achieve optimal results in construction. Furthermore, by examining the effectiveness of a multi-objective optimization method as an adaptive design driver, this paper contributes to the development of novel computational strategies that can enhance the flexibility and adaptability of robotic construction systems.