NeCTAr and RASoC: Tale of Two Class SoCs for Language Model Interference and Robotics in Intel 16
Viansa Schmulbach, Jason Kim, Ethan Gao, Nikhil Jha, Ethan Wu, Oliver Yu, Ben Oliveau, Xiangwei Kong, Brendan Roberts, Connor McMahon, Lixiang Yin, Vamber Yang, Brendan Brenner, George Moujaes, Boyu Hao, Lucy Revina, Kevin Anderson, Bryan Ngo, Yufeng Chi, Hongyi Huang
- Year
- 2024
- Citations
- 5
Abstract
This paper introduces NeCTAr (Near-Cache Transformer Accelerator), a 16nm heterogeneous multicore RISC-V SoC for sparse and dense machine learning kernels with both near-core and near-memory accelerators. A prototype chip runs at 400MHz at 0.85V and performs matrix-vector multiplications with 109 GOPs/W. The effectiveness of the design is demonstrated by running inference on a sparse language model, ReLU-Llama.
Keywords
Related papers
Statistical Learning Theory
Yuhai Wu, Vladimir Vapnik
1999
Artificial intelligence: a modern approach
1995
Fractional Differential Equations
Igor Podlubný
2025
Applied Nonlinear Control
Jean-Jacques Slotine, Weiping Li
1991