Hardware-aware Graph Neural Networks prunning for embedded event-based vision

Abstract

Event-based cameras are gaining popularity as the sensor of choice for mobile robotics, due to their high performance in dynamic environments. However, these applications require efficient real-time data processing with low latency and power consumption. One strategy to meet these stringent requirements is hardware acceleration of efficient algorithms that preserve the temporal sparsity of event data. In this work, we propose an optimization strategy for Graph Convolutional Neural Networks models aimed at adapting their architecture to the limited resources of embedded heterogeneous FPGA platforms. Our method incorporates hardware-aware pruning and quantization, taking into account the trade-off between on-chip memory savings and inference accuracy. Strategic exploration of the design space with Fine Grid Search and Greedy layer-wise Iterative Deepening Search methods enables flexible adaptation of the model architecture to the target platform. Our approach was evaluated across various network configurations and multiple datasets, resulting in BRAM memory reductions of 28.8% for CIFAR-10 (with a 1.65% decrease in accuracy), 31.4% for MNIST-DVS (accuracy drop of 3.55%), and 26.5% for N-Caltech101 (with a 5.18% accuracy reduction).