SpamHD: Memory-Efficient Text Spam Detection using Brain-Inspired Hyperdimensional Computing

Abstract

Brain-inspired hyperdimensional Computing (HDC) leverages the mathematical properties of high-dimensional vectors (hypervectors) which show remarkable agreement with how brain functions. Hypervectors (HVs) are high-dimensional (e.g., 10,000 dimensions), holographic, and (pseudo)random with independent and identically distributed (i.i.d) components. Recently, HDC has demonstrated promising capability in a wide range of applications such as robotics, bio-medical signal processing, and genome sequencing. Text spam detection is a classic natural language processing (NLP) task that is usually solved using machine learning methods associated with data preprocessing techniques such as tokenization. In this paper, we develop a memory-efficient text spam detection approach called SpamHD based on HDC methods. In addition to the conventional tokenization-based approach, we also develop a tokenization-free HDC approach with N-gram encoding. Experimental results on three real-world spam datasets (Hotel review, SMS text, and YouTube comments) show that SpamHD is able achieve similar or even outperform baseline tokenization-based learning methods, but with significantly less storage requirements (30X-115X model size reduction). Further, we perform a design space exploration for SpamHD by tuning the number of dimensions of HVs and encoding methods, and evaluate the impact of such design parameters on accuracy and memory requirements.