Opportunities for Analog Coding in Emerging Memory Systems

The exponential growth in data generation and large-scale data analysis creates an unprecedented need for inexpensive, low-latency, and high-density information storage. This need has motivated significant research into multi-level memory systems that can store multiple bits of information per device. Although both the memory state of these devices and much of the data they store are intrinsically analog-valued, both are quantized for use with digital systems and discrete error correcting codes. Using phase change memory as a prototypical multi-level storage technology, we herein demonstrate that analog-valued devices can achieve higher capacities when paired with analog codes. Further, we find that storing analog signals directly through joint-coding can achieve low distortion with reduced coding complexity. By jointly optimizing for signal statistics, device statistics, and a distortion metric, finite-length analog encodings can perform comparable to digital systems with asymptotically infinite large encodings. These results show that end-to-end analog memory systems have not only the potential to reach higher storage capacities than discrete systems, but also to significantly lower coding complexity, leading to faster and more energy efficient storage.

[1]  Dave Evans,et al.  How the Next Evolution of the Internet Is Changing Everything , 2011 .

[2]  Michael Gastpar,et al.  To code, or not to code: lossy source-channel communication revisited , 2003, IEEE Trans. Inf. Theory.

[3]  L. A. Lastras-Montao,et al.  Rewritable storage channels , 2008, 2008 International Symposium on Information Theory and Its Applications.

[4]  Eero P. Simoncelli,et al.  Image quality assessment: from error visibility to structural similarity , 2004, IEEE Transactions on Image Processing.

[5]  Suguru Arimoto,et al.  An algorithm for computing the capacity of arbitrary discrete memoryless channels , 1972, IEEE Trans. Inf. Theory.

[6]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[7]  Chung Lam,et al.  Experimental demonstration of array-level learning with phase change synaptic devices , 2013, 2013 IEEE International Electron Devices Meeting.

[8]  E. Eleftheriou,et al.  Drift-Tolerant Multilevel Phase-Change Memory , 2011, 2011 3rd IEEE International Memory Workshop (IMW).

[9]  Kenneth Rose,et al.  Optimal mappings for joint source channel coding , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[10]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[11]  Shimeng Yu,et al.  Synaptic electronics: materials, devices and applications , 2013, Nanotechnology.

[12]  M. Breitwisch,et al.  Novel Lithography-Independent Pore Phase Change Memory , 2007, 2007 IEEE Symposium on VLSI Technology.

[13]  Shimeng Yu,et al.  Metal–Oxide RRAM , 2012, Proceedings of the IEEE.

[14]  Mohammad M. Mansour,et al.  A 640-Mb/s 2048-bit programmable LDPC decoder chip , 2006, IEEE Journal of Solid-State Circuits.

[15]  Y.C. Chen,et al.  Write Strategies for 2 and 4-bit Multi-Level Phase-Change Memory , 2007, 2007 IEEE International Electron Devices Meeting.

[16]  M. Breitwisch Phase Change Memory , 2008, 2008 International Interconnect Technology Conference.

[17]  Lav R. Varshney,et al.  Optimal Information Storage in Noisy Synapses under Resource Constraints , 2006, Neuron.

[18]  Wonyong Sung,et al.  Rate-0.96 LDPC Decoding VLSI for Soft-Decision Error Correction of NAND Flash Memory , 2014, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[19]  Gert Cauwenberghs,et al.  Event-driven contrastive divergence for spiking neuromorphic systems , 2013, Front. Neurosci..

[20]  H.-S. Philip Wong,et al.  Effect of Wordline/Bitline Scaling on the Performance, Energy Consumption, and Reliability of Cross-Point Memory Array , 2013, JETC.

[21]  Sergio Verdú,et al.  Lossy Joint Source-Channel Coding in the Finite Blocklength Regime , 2012, IEEE Transactions on Information Theory.

[22]  David Minnen,et al.  Full Resolution Image Compression with Recurrent Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[24]  Tara N. Sainath,et al.  Deep Neural Networks for Acoustic Modeling in Speech Recognition , 2012 .

[25]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[26]  C. Hagleitner,et al.  Device, circuit and system-level analysis of noise in multi-bit phase-change memory , 2010, 2010 International Electron Devices Meeting.

[27]  Eero P. Simoncelli,et al.  Efficient coding of natural images with a population of noisy Linear-Nonlinear neurons , 2011, NIPS.

[28]  Luca Crippa,et al.  A 4Gb 2b/cell NAND Flash Memory with Embedded 5b BCH ECC for 36MB/s System Read Throughput , 2006, 2006 IEEE International Solid State Circuits Conference - Digest of Technical Papers.