Mixed-signal in-memory matrix-vector multiplication for ultra-low power embedded machine learning

Overview & Synopsis

The applications for embedded artificial intelligence are numerous and cover multiple domains, such as consumer electronics, home automation, health, and industry. They require dedicated chips bringing intelligence close to the sensor while maintaining a low energy consumption. Although many types of neural networks (NN) exist, they all rely on the same basic computations which are Matrix-Vector Multiplications (MVM) composed of Multiply-and-Accumulate (MAC) operations. Optimizing the energy efficiency of MAC operations is a great lever to reduce global power consumption. In a classic Von Neumann architecture, the limitation implied by data access caps the efficiency at 10 TOPS/W considering a 50 fJ/byte energy consumption for data movement. In-memory computing (IMC) helps reduce the energy overhead for accessing data by processing them close to where they are stored. This thesis analyses the state-of-the-art NN architectures and the works for Voice Activity Detection (VAD) and Keyword Spotting (KWS), to show that energy consumption and accuracy are more important parameters than throughput for embedded applications. Furthermore, analysis of the state-of-the-art of IMC shows that the available time to perform NN operations can be advantageously leveraged. This work presents a time- and current-based analog IMC concept, where current sources charge/discharge a capacitive line during a time pondered by the product of two numbers, therefore performing multi-bit MAC operations through time. An implementation of the proposed architecture in a 28 nm FDSOI technology is presented. The integrated circuit prototype integrates 4 neurons with 100 inputs and 5-bit inputs and weights. The structure performs the multi-bit MVM using the proposed time- and current-based analogue IMC method within a maximum latency of 4.5 µs, perfectly suitable with most embedded applications. The measured energy efficiency allows envisioning > 50 TOPS/W if deployed over a 100-neuron array.