Learning a neural response metric for retinal prosthesis

Retinal prostheses for treating incurable blindness are designed to electrically stimulate surviving retinal neurons, causing them to send artificial visual signals to the brain. However, electrical stimulation generally cannot precisely reproduce typical patterns of neural activity in the retina. Therefore, an electrical stimulus must be selected so as to produce a neural response as close as possible to the desired response. This setting requires a technique for computing the distance between a desired response and an achievable response that is meaningful in terms of the visual signal being conveyed. We propose a method to learn a metric on neural responses directly from recorded light responses of a population of retinal ganglion cells (RGCs) in the primate retina. The learned metric produces a measure of similarity of RGC population responses that accurately reflects the similarity of visual inputs. Using data from electrical stimulation experiments, we demonstrate that the learned metric leads to improvements in the performance of a retinal prosthesis.

[1]  D. Mastronarde Correlated firing of cat retinal ganglion cells. I. Spontaneously active inputs to X- and Y-cells. , 1983, Journal of neurophysiology.

[2]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[3]  William Bialek,et al.  Reading a Neural Code , 1991, NIPS.

[4]  William M. Rabinowitz,et al.  Better speech recognition with cochlear implants , 1991, Nature.

[5]  J. Roddey,et al.  Information theoretic analysis of dynamical encoding by filiform mechanoreceptors in the cricket cercal system. , 1996, Journal of neurophysiology.

[6]  J. Victor,et al.  Nature and precision of temporal coding in visual cortex: a metric-space analysis. , 1996, Journal of neurophysiology.

[7]  William Bialek,et al.  Spikes: Exploring the Neural Code , 1996 .

[8]  Jonathan D. Victor,et al.  Metric-space analysis of spike trains: theory, algorithms and application , 1998, q-bio/0309031.

[9]  Michael J. Berry,et al.  The structure and precision of retinal spike trains. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[10]  G B Stanley,et al.  Reconstruction of Natural Scenes from Ensemble Responses in the Lateral Geniculate Nucleus , 1999, The Journal of Neuroscience.

[11]  Yoav Freund,et al.  A Short Introduction to Boosting , 1999 .

[12]  E J Chichilnisky,et al.  A simple white noise analysis of neuronal light responses , 2001, Network.

[13]  Mark C. W. van Rossum,et al.  A Novel Spike Distance , 2001, Neural Computation.

[14]  F. Mechler,et al.  Neural coding of spatial phase in V1 of the macaque monkey. , 2003, Journal of neurophysiology.

[15]  A.M. Litke,et al.  What does the eye tell the brain?: Development of a system for the large scale recording of retinal output activity , 2003, 2003 IEEE Nuclear Science Symposium. Conference Record (IEEE Cat. No.03CH37515).

[16]  Andrew B Schwartz,et al.  Cortical neural prosthetics. , 2004, Annual review of neuroscience.

[17]  Yoram Singer,et al.  Online and batch learning of pseudo-metrics , 2004, ICML.

[18]  K. Deisseroth,et al.  Millisecond-timescale, genetically targeted optical control of neural activity , 2005, Nature Neuroscience.

[19]  Jonathan D Victor,et al.  Spike train metrics , 2005, Current Opinion in Neurobiology.

[20]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[21]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  E. Chichilnisky,et al.  Fidelity of the ensemble code for visual motion in primate retina. , 2005, Journal of neurophysiology.

[23]  Yann LeCun,et al.  Dimensionality Reduction by Learning an Invariant Mapping , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[24]  G D Field,et al.  Information processing in the primate retina: circuitry and coding. , 2007, Annual review of neuroscience.

[25]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[26]  Eero P. Simoncelli,et al.  Spatio-temporal correlations and visual signalling in a complete neuronal population , 2008, Nature.

[27]  Xue Han,et al.  Prosthetic systems for therapeutic optical activation and silencing of genetically targeted neurons , 2008, SPIE BiOS.

[28]  E. Chichilnisky,et al.  High-Resolution Electrical Stimulation of Primate Retina for Epiretinal Implant Design , 2008, The Journal of Neuroscience.

[29]  Jonathon Shlens,et al.  The Structure of Large-Scale Synchronized Firing in Primate Retina , 2009, The Journal of Neuroscience.

[30]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[31]  Alfred Stett,et al.  Subretinal electronic chips allow blind patients to read letters and combine them to words , 2010, Proceedings of the Royal Society B: Biological Sciences.

[32]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[33]  Marcelo O. Magnasco,et al.  A Fast p Spike Alignment Metric , 2010, Neural Computation.

[34]  Marcelo O. Magnasco,et al.  A Fast p Spike Alignment Metric , 2009, Neural Computation.

[35]  Jonathon Shlens,et al.  Correlated firing among major ganglion cell types in primate retina , 2011, The Journal of physiology.

[36]  B. Wilhelm,et al.  Spatial resolution and perception of patterns mediated by a subretinal 16-electrode array in patients blinded by hereditary retinal dystrophies. , 2011, Investigative ophthalmology & visual science.

[37]  Martin J. Wainwright,et al.  Randomized smoothing for (parallel) stochastic optimization , 2012, 2012 IEEE 51st IEEE Conference on Decision and Control (CDC).

[38]  Jessy D. Dorn,et al.  Interim results from the international trial of Second Sight's visual prosthesis. , 2012, Ophthalmology.

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Angelika Braun,et al.  Artificial vision with wirelessly powered subretinal electronic implant alpha-IMS , 2013, Proceedings of the Royal Society B: Biological Sciences.

[41]  E. Chichilnisky,et al.  Spatially Patterned Electrical Stimulation to Enhance Resolution of Retinal Prostheses , 2014, The Journal of Neuroscience.

[42]  E. Chichilnisky,et al.  High-Fidelity Reproduction of Spatiotemporal Visual Signals for Retinal Prosthesis , 2014, Neuron.

[43]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[44]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  J. Barrett,et al.  Optogenetic approaches to retinal prosthesis , 2014, Visual Neuroscience.

[46]  Yang Song,et al.  Learning Fine-Grained Image Similarity with Deep Ranking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[47]  Georges Goetz,et al.  Recognizing retinal ganglion cells in the dark , 2015, NIPS.

[48]  Ronen Segev,et al.  A thesaurus for a neural population code , 2015, eLife.

[49]  James Philbin,et al.  FaceNet: A unified embedding for face recognition and clustering , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Zachary Chase Lipton A Critical Review of Recurrent Neural Networks for Sequence Learning , 2015, ArXiv.

[51]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[52]  E. Callaway,et al.  Anatomical Identification of Extracellularly Recorded Cells in Large-Scale Multielectrode Recordings , 2015, The Journal of Neuroscience.

[53]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[54]  Michael J. Berry,et al.  Error-Robust Modes of the Retinal Population Code , 2016, PLoS Comput. Biol..

[55]  Surya Ganguli,et al.  Deep Learning Models of the Retinal Response to Natural Scenes , 2017, NIPS.

[56]  Kihyuk Sohn,et al.  Improved Deep Metric Learning with Multi-class N-pair Loss Objective , 2016, NIPS.

[57]  Silvio Savarese,et al.  Deep Metric Learning via Lifted Structured Feature Embedding , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  Liam Paninski,et al.  Multilayer Recurrent Network Models of Primate Retinal Ganglion Cell Responses , 2016, ICLR.

[59]  John P. Cunningham,et al.  Electrical stimulus artifact cancellation and neural spike detection on large multi-electrode arrays , 2016, bioRxiv.

[60]  Thierry Mora,et al.  Blindfold learning of an accurate neural metric , 2017, Proceedings of the National Academy of Sciences.