A Low-Latency Framework With Algorithm-Hardware Co-Optimization for 3-D Point Cloud

Abstract

As an important type of 3D representation, the point cloud is widely used in many applications, such as autonomous driving, AR/VR, and intelligent robots, which require real-time interactions with humans. However, the sparsity of 3D point cloud data leads to severe computational inefficiency when being processed by 2D data processors, posing a huge challenge for hardware acceleration. In this brief, we aim at solving the inefficiency problem by algorithm-hardware co-optimization. Firstly, a lightweight network, named LPN, is proposed for point cloud data classification, which is <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$30\times $ </tex-math></inline-formula> smaller than pointnet and still has comparable accuracy. Secondly, a reconfigurable computing core, named RCC, together with an adaptive dataflow, is developed to support different layers of the LPN. Specifically, to accelerate memory-intensive layers, a partially-parallel computing scheme is introduced to minimize the on-chip memory requirements and DRAM accesses. Finally, based on the above innovations, a low-latency accelerator is proposed to realize real-time computation for the point cloud, which is implemented on the Xilinx Kintex UltraScale KCU150 FPGA board. Experimental results show that it achieves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$1.5\times $ </tex-math></inline-formula> throughput improvement compared with the state-of-the-art works, and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$35\times $ </tex-math></inline-formula> speedup over Intel Xeon Gold 6148 CPU, demonstrating the superiority of the proposed method. The code of LPN is available from <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/snowsil/LPN-model-for-3D-classifification</uri> .