Stochastic decoding of low-density parity-check codes

Low-Density Parity-Check (LDPC) codes are one of the most powerful classes of error-control codes known to date. These codes have been considered for many recent digital communication applications. In this dissertation, we propose stochastic decoding of state-of-the-art LDPC codes and demonstrate it as a competitive approach to practical LDPC decoding algorithms.In stochastic decoding, probabilities are represented as streams of random bits using Bernoulli sequences in which the information is contained in the statistics of the bit stream. This representation results in low hardware-complexity processing nodes that perform computationally-intensive operations. However, stochastic decoding is prone to the acute problem of latching. This problem is caused by correlated bit streams within cycles in the code's factor graph, and significantly deteriorates the performance of stochastic LDPC decoders.We propose edge memories, tracking forecast memories, and majority-based tracking forecast memories to address the latching problem. These units efficiently extract the evolving statistics of stochastic bit streams and rerandomize them to disrupt latching. To the best of our knowledge, these methods are the first successful methods for stochastic decoding of state-of-the-art LDPC codes.We present novel decoder architectures and report on several hardware implementations. The most advanced reported implementation is a stochastic decoder that decodes the (2048,1723) LDPC code from the IEEE 802.3an standard. To the best of our knowledge, this decoder is the most silicon area-efficient and, with a maximum core throughput of 61.3 Gb/s, is one of the fastest fully parallel soft-decision LDPC decoders reported in the literature. We demonstrate the performance of this decoder in low bit-error-rate regimes.In addition to stochastic LDPC decoding, we propose the novel application of the stochastic approach for joint decoding of LDPC codes and partial-response channels that are considered in practical magnetic recording applications. Finally, we investigate the application of the stochastic approach for decoding linear block codes with high-density parity-check matrices on factor graphs. We consider Reed-Solomon, Bose-Chaudhuri-Hocquenghem, and block turbo codes. %%%%A ce jour, les codes Low-Density Parity-Check (LDPC) font partie des codes correcteurs d'erreurs les plus performants. Ces codes sont inclus dans differents standards de communications numeriques. Dans ce manuscrit, nous proposons d'utiliser le decodage stochastique pour les codes LDPC. D'autre part, nous demontrons que pour les codes LDPC, le decodage stochastique represente une alternative realiste aux algorithmes de decodage existants.Dans le processus de decodage stochastique, les probabilites sont representees sous forme de sequences de Bernoulli. L'information est contenue dans la statistique de ces flux binaires aleatoires. Cette representation particuliere permet d'executer des calculs intensifs avec une faible complexite materielle. Cependant le decodage stochastique…

[1]  Shie Mannor,et al.  Majority-Based Tracking Forecast Memories for Stochastic LDPC Decoding , 2010, IEEE Transactions on Signal Processing.

[2]  Masoud Ardakani,et al.  Optimum Linear LLR Calculation for Iterative Decoding on Fading Channels , 2007, 2007 IEEE International Symposium on Information Theory.

[3]  Bruce F. Cockburn,et al.  A scalable LDPC decoder ASIC architecture with bit-serial message exchange , 2008, Integr..

[4]  Tinoosh Mohsenin,et al.  High-Throughput LDPC Decoders Using A Multiple Split-Row Method , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[5]  Brian R. Gaines Advances in information systems science , 1969 .

[6]  Frank R. Kschischang,et al.  A 3.3-Gbps bit-serial block-interlaced min-sum LDPC decoder in 0.13-μm CMOS , 2007, 2007 IEEE Custom Integrated Circuits Conference.

[7]  Amir H. Banihashemi,et al.  Improving belief propagation on graphs with cycles , 2004, IEEE Communications Letters.

[8]  Christian Schlegel,et al.  Trellis and turbo coding , 2004 .

[9]  Minyue Fu Stochastic models for turbo decoding , 2005, IEEE International Conference on Communications, 2005. ICC 2005. 2005.

[10]  Frank R. Kschischang,et al.  Block-Interlaced LDPC Decoders With Reduced Interconnect Complexity , 2008, IEEE Transactions on Circuits and Systems II: Express Briefs.

[11]  Evangelos Eleftheriou,et al.  Regular and irregular progressive edge-growth tanner graphs , 2005, IEEE Transactions on Information Theory.

[12]  Shie Mannor,et al.  A Relaxed Half-Stochastic Iterative Decoder for LDPC Codes , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[13]  Naresh R. Shanbhag,et al.  High-throughput LDPC decoders , 2003, IEEE Trans. Very Large Scale Integr. Syst..

[14]  Zhongfeng Wang,et al.  Low-Complexity High-Speed Decoder Design for Quasi-Cyclic LDPC Codes , 2007, IEEE Transactions on Very Large Scale Integration (VLSI) Systems.

[15]  C. Winstead Stochastic Iterative Decoding on Factor Graphs , 2003 .

[16]  Shie Mannor,et al.  Bidirectional interleavers for LDPC decoders using transmission gates , 2009, 2009 IEEE Workshop on Signal Processing Systems.

[17]  Krishna R. Narayanan,et al.  Iterative soft decoding of Reed-Solomon codes , 2004, IEEE Communications Letters.

[18]  Lara Dolecek,et al.  Design of LDPC decoders for improved low error rate performance: quantization and algorithm choices , 2009, IEEE Transactions on Communications.

[19]  Christophe Jégo,et al.  Turbo decoding of product codes using adaptive belief propagation , 2009, IEEE Transactions on Communications.

[20]  Mohammed Atiquzzaman,et al.  VLSI Architectures for Layered Decoding for Irregular LDPC Codes of WiMax , 2007, 2007 IEEE International Conference on Communications.

[21]  Shie Mannor,et al.  An Area-Efficient FPGA-Based Architecture for Fully-Parallel Stochastic LDPC Decoding , 2007, 2007 IEEE Workshop on Signal Processing Systems.

[22]  Vincent C. Gaudet,et al.  Iterative decoding using stochastic computation , 2003 .

[23]  D. Young,et al.  A Survey of Numerical Mathematics , 1988 .

[24]  Jean-Luc Danger,et al.  Lambda-Min Decoding Algorithm of Regular and Irregular LDPC Codes , 2003 .

[25]  Marc P. C. Fossorier,et al.  A modified weighted bit-flipping decoding of low-density Parity-check codes , 2004, IEEE Communications Letters.

[26]  Martin J. Wainwright,et al.  An Efficient 10GBASE-T Ethernet LDPC Decoder Design With Low Error Floors , 2010, IEEE Journal of Solid-State Circuits.

[27]  Martin J. Wainwright,et al.  A 47 Gb/s LDPC decoder with improved low error rate performance , 2009, 2009 Symposium on VLSI Circuits.

[28]  Yan Li,et al.  Improved min-sum decoding algorithms for irregular LDPC codes , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[29]  Sang Joon Kim,et al.  A Mathematical Theory of Communication , 2006 .

[30]  Rüdiger L. Urbanke,et al.  The capacity of low-density parity-check codes under message-passing decoding , 2001, IEEE Trans. Inf. Theory.

[31]  T. Hanyu,et al.  3.2-Gb/s 1024-b rate-1/2 LDPC decoder chip using a flooding-type update-schedule algorithm , 2007, 2007 50th Midwest Symposium on Circuits and Systems.

[32]  C. Plett,et al.  A 0.18-$muhbox m$CMOS Analog Min-Sum Iterative Decoder for a (32,8) Low-Density Parity-Check (LDPC) Code , 2006, IEEE Journal of Solid-State Circuits.

[33]  Frank R. Kschischang,et al.  A bit-serial approximate min-sum LDPC decoder and FPGA implementation , 2006, 2006 IEEE International Symposium on Circuits and Systems.

[34]  Jinghu Chen,et al.  Density evolution for two improved BP-Based decoding algorithms of LDPC codes , 2002, IEEE Communications Letters.

[35]  F. Chiaraluce,et al.  Extended Hamming product codes analytical performance evaluation for low error rate applications , 2004, IEEE Transactions on Wireless Communications.

[36]  Marcian Cirstea,et al.  Stochastic implementation of motor controllers , 2002, Industrial Electronics, 2002. ISIE 2002. Proceedings of the 2002 IEEE International Symposium on.

[37]  Tong Zhang,et al.  Joint (3,k)-regular LDPC code and decoder/encoder design , 2004, IEEE Transactions on Signal Processing.

[38]  Shie Mannor,et al.  Tracking Forecast Memories in stochastic decoders , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[39]  Shie Mannor,et al.  Online calibrated forecasts: Memory efficiency versus universality for learning in games , 2006, Machine Learning.

[40]  Masoud Ardakani,et al.  Gear-shift decoding , 2006, IEEE Transactions on Communications.

[41]  Tinoosh Mohsenin,et al.  Split-Row: A Reduced Complexity, High Throughput LDPC Decoder Architecture , 2006, 2006 International Conference on Computer Design.

[42]  Achilleas Anastasopoulos,et al.  A comparison between the sum-product and the min-sum iterative detection algorithms based on density evolution , 2001, GLOBECOM'01. IEEE Global Telecommunications Conference (Cat. No.01CH37270).

[43]  Sae-Young Chung,et al.  On the design of low-density parity-check codes within 0.0045 dB of the Shannon limit , 2001, IEEE Communications Letters.

[44]  V. Anantharam,et al.  Design of LDPC Decoders for Low Error Rate Performance , 2008 .

[45]  W.J. Gross,et al.  Stochastic Implementation of LDPC Decoders , 2005, Conference Record of the Thirty-Ninth Asilomar Conference onSignals, Systems and Computers, 2005..

[46]  Venkat Anantharam,et al.  Iterative decoder architectures , 2003, IEEE Commun. Mag..

[47]  Vincent C. Gaudet,et al.  Decoder IC with a Configurable Interleaver , 2003 .

[48]  Ramesh Pyndiah,et al.  Near-optimum decoding of product codes: block turbo codes , 1998, IEEE Trans. Commun..

[49]  Shie Mannor,et al.  Stochastic decoding of LDPC codes , 2006, IEEE Communications Letters.

[50]  Shie Mannor,et al.  Fully Parallel Stochastic LDPC Decoders , 2008, IEEE Transactions on Signal Processing.

[51]  Tinoosh Mohsenin,et al.  A Low-Complexity Message-Passing Algorithm for Reduced Routing Congestion in LDPC Decoders , 2010, IEEE Transactions on Circuits and Systems I: Regular Papers.

[52]  Shie Mannor,et al.  Survey of Stochastic Computation on Factor Graphs , 2007, 37th International Symposium on Multiple-Valued Logic (ISMVL'07).

[53]  Warren J. Gross,et al.  Reduced-latency stochastic decoding of LDPC codes over GF(q) , 2010, 2010 European Wireless Conference (EW).

[54]  Ajay Dholakia,et al.  Reduced-complexity decoding of LDPC codes , 2005, IEEE Transactions on Communications.

[55]  R.R. Harrison,et al.  CMOS analog MAP decoder for (8,4) Hamming code , 2004, IEEE Journal of Solid-State Circuits.

[56]  J.K. Wolf,et al.  Distance Spectra For PRML Channels , 1997, 1997 IEEE International Magnetics Conference (INTERMAG'97).

[57]  Shie Mannor,et al.  Tracking Forecast Memories for Stochastic Decoding , 2011, J. Signal Process. Syst..

[58]  D.J.C. MacKay,et al.  Good error-correcting codes based on very sparse matrices , 1997, Proceedings of IEEE International Symposium on Information Theory.

[59]  Frank R. Kschischang,et al.  Multi-Gbit/sec low density parity check decoders with reduced interconnect complexity , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[60]  Gwan S. Choi,et al.  A Parallel VLSI Architecture for Layered Decoding for Array LDPC Codes , 2007, 20th International Conference on VLSI Design held jointly with 6th International Conference on Embedded Systems (VLSID'07).

[61]  Fabrice Seguin,et al.  Analog slice turbo decoding , 2005, 2005 IEEE International Symposium on Circuits and Systems.

[62]  Ming Jiang,et al.  An improvement on the modified weighted bit flipping decoding algorithm for LDPC codes , 2005, IEEE Communications Letters.

[63]  Pascal Urard,et al.  Based on 64800b LDPC and BCH Codes , 2005 .

[64]  Niclas Wiberg,et al.  Codes and Decoding on General Graphs , 1996 .

[65]  Vincent C. Gaudet,et al.  Stochastic iterative decoders , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[66]  Chaitali Chakrabarti,et al.  Design and Analysis of LDPC Decoders for Software Defined Radio , 2007, 2007 IEEE Workshop on Signal Processing Systems.

[67]  Bo Zhu,et al.  Stochastic Decoding of Linear Block Codes With High-Density Parity-Check Matrices , 2008, IEEE Transactions on Signal Processing.

[68]  Joachim Hagenauer,et al.  WA 21.5 An Analog 0.25∝m BiCMOS Tailbiting MAP Decoder , 2000 .

[69]  Robert G. Gallager,et al.  Low-density parity-check codes , 1962, IRE Trans. Inf. Theory.

[70]  Giulio Colavolpe,et al.  On the application of factor graphs and the sum-product algorithm to ISI channels , 2005, IEEE Transactions on Communications.

[71]  Paul H. Siegel,et al.  Joint message-passing decoding of LDPC Codes and partial-response channels , 2002, IEEE Trans. Inf. Theory.

[72]  David J. C. MacKay,et al.  Good Codes Based on Very Sparse Matrices , 1995, IMACC.

[73]  M. Helfenstein,et al.  All–analog decoder for a binary (18,9,5) tail–biting trellis code , 1999, Proceedings of the 25th European Solid-State Circuits Conference.

[74]  James M. Ortega,et al.  Iterative solution of nonlinear equations in several variables , 2014, Computer science and applied mathematics.

[75]  Radford M. Neal,et al.  Near Shannon limit performance of low density parity check codes , 1996 .

[76]  Howard C. Card,et al.  Stochastic Neural Computation I: Computational Elements , 2001, IEEE Trans. Computers.

[77]  Amir H. Banihashemi,et al.  A Differential Binary Message-Passing LDPC Decoder , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[78]  Shie Mannor,et al.  Stochastic Decoding of LDPC Codes over GF(q) , 2013, IEEE Transactions on Communications.

[79]  C. Kelley Iterative Methods for Linear and Nonlinear Equations , 1987 .

[80]  A. Glavieux,et al.  Near Shannon limit error-correcting coding and decoding: Turbo-codes. 1 , 1993, Proceedings of ICC '93 - IEEE International Conference on Communications.

[81]  X. Jin Factor graphs and the Sum-Product Algorithm , 2002 .

[82]  Krishna R. Narayanan,et al.  Iterative Soft-Input Soft-Output Decoding of Reed–Solomon Codes by Adapting the Parity-Check Matrix , 2005, IEEE Transactions on Information Theory.

[83]  A. Blanksby,et al.  A 690-mW 1-Gb/s 1024-b, rate-1/2 low-density parity-check code decoder , 2001, IEEE J. Solid State Circuits.

[84]  Christophe Jégo,et al.  Turbo Decoding of Product Codes based on the Modified Adaptive Belief Propagation Algorithm , 2007, 2007 IEEE International Symposium on Information Theory.

[85]  A. Neviani,et al.  A 0.35-/spl mu/m CMOS analog turbo decoder for the 40-bit rate 1/3 UMTS channel code , 2005, IEEE Journal of Solid-State Circuits.

[86]  Thierry Paul,et al.  Quantum computation and quantum information , 2007, Mathematical Structures in Computer Science.

[87]  Stephen B. Wicker,et al.  Applications of Error-Control Coding , 1998, IEEE Trans. Inf. Theory.

[88]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[89]  Lara Dolecek,et al.  Lowering LDPC Error Floors by Postprocessing , 2008, IEEE GLOBECOM 2008 - 2008 IEEE Global Telecommunications Conference.

[90]  Stephen G. Wilson,et al.  Multi-Gbps FPGA-Based Low Density Parity Check (LDPC) Decoder Design , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.

[91]  Shu Lin,et al.  A class of low-density parity-check codes constructed based on Reed-Solomon codes with two information symbols , 2003, IEEE Communications Letters.

[92]  Vincent C. Gaudet,et al.  A degree-matched check node approximation for LDPC decoding , 2005, Proceedings. International Symposium on Information Theory, 2005. ISIT 2005..

[93]  Dariush Divsalar,et al.  The Development of Turbo and LDPC Codes for Deep-Space Applications , 2007, Proceedings of the IEEE.

[94]  C.-J. Richard Shi,et al.  Sliced Message Passing: High Throughput Overlapped Decoding of High-Rate Low-Density Parity-Check Codes , 2008, IEEE Transactions on Circuits and Systems I: Regular Papers.

[95]  Jung-Fu Cheng,et al.  Turbo Decoding as an Instance of Pearl's "Belief Propagation" Algorithm , 1998, IEEE J. Sel. Areas Commun..

[96]  Mohammad M. Mansour,et al.  A 640-Mb/s 2048-bit programmable LDPC decoder chip , 2006, IEEE Journal of Solid-State Circuits.