Accurate Inference With Inaccurate RRAM Devices: A Joint Algorithm-Design Solution

• Main Authors: Gouranga Charan, Abinash Mohanty, Xiaocong Du, Gokul Krishnan, Rajiv V. Joshi, Yu Cao
• Format: Article
• Language: English
• Published: IEEE 2020-01-01
• Series: IEEE Journal on Exploratory Solid-State Computational Devices and Circuits
• Subjects: Convolution neural networks; device nonidealities; model robustness; neuromorphic computing; random sparse adaptation (RSA); resistive random access memory (RRAM)
• Online Access: https://ieeexplore.ieee.org/document/9069242/

Resistive random access memory (RRAM) is a promising technology for energy-efficient neuromorphic accelerators. However, when a pretrained deep neural network (DNN) model is programmed to an RRAM array for inference, the model suffers from accuracy degradation due to RRAM nonidealities, such as device variations, quantization error, and stuck-at-faults. Previous solutions involving multiple read-verify-write (R-V-W) to the RRAM cells require cell-by-cell compensation and, thus, an excessive amount of processing time. In this article, we propose a joint algorithm-design solution to mitigate the accuracy degradation. We first leverage knowledge distillation (KD), where the model is trained with the RRAM nonidealities to increase the robustness of the model under device variations. Furthermore, we propose random sparse adaptation (RSA), which integrates a small on-chip memory with the main RRAM array for postmapping adaptation. Only the on-chip memory is updated to recover the inference accuracy. The joint algorithm-design solution achieves the state-of-the-art accuracy of 99.41% for MNIST (LeNet-5) and 91.86% for CIFAR-10 (VGG-16) with up to 5% parameters as overhead while providing a 15-150× speedup compared with R-V-W.