Achieving Robust Neural Representations : An Account Of Repetition Suppression

An important source of evidence concerning rapid adaptation and learning in the brain is the robust phenomenon of repetition suppression—the long lasting and item-specific decrease in neural activity with repeated exposure to an item, yielding sparser, sharper representations. Existing accounts of repetition suppression are informal and do little more than describe the phenomenon. We explore the hypothesis that repetition suppression arises from an unsupervised learning mechanism that reduces sensitivity to noise by increasing the item-specific gain of neural responses, in conjunction with the assumption that neurons are biased toward infrequent activity. This hypothesis explains key experimental observations concerning changes in neural representation with mere repetition of stimuli, regardless of task relevance. Additionally, this hypothesis explains related data concerning improved discriminability and noise robustness of individual neurons due to practice on a specific task. A widespread and robust finding in primate electrophysiology is that neural responses decrease over repeated exposure to a stimulus (e.g., [2, 13, 16]). Decreased activation is also observed in human imaging studies (e.g., [11]) and a reduction of waveform amplitude is observed in ERP studies [14]. This repetition suppression (or RS) effect is often interpreted to reflect an increase in neural efficiency and likely mediates the psychological phenomenon of repetition priming [16, 11], in which prior presentation of a stimulus leads to more efficient processing of the same stimulus in the future. Repetition suppression is also referred to as stimulus-specific adaptation in the literature. Key findings in the literature are as follows. (1) RS involves sharpening the neural representation, i.e., reducing overall neural activity and decreasing the number of neurons involved in representing an item [2, 13, 16]. Although fewer neurons are active for an item, a small fraction show an increase in response [13]. (2) RS is item specific, not general habituation [2, 13]. (3) RS is long lasting; it is observed even when the two repetitions are separated by 150 intervening items and delays up to 24 hours [6, 13]. (4) RS is graded, showing a continual reduction in firing rate with each presentation, plateauing at about half of the initial firing rate [6, 13]. (5) RS has been observed in many cortical regions, particularly inferior temporal and prefrontal cortex. However, RS is not omnipresent, e.g., it is not found in V1 [8], and it is found for words but not pseudowords in left posterior fusiform gyrus [3]. (6) RS depends merely on repetition, not behavioral significance [16]. This final characteristic—that RS occurs even with passive viewing, when no response is required and when the stimulus is not associated with a task or a reward—distinguishes RS in principle from adaptations that occur with skill learning. However, Rainer and Miller [12] have reported intriguing similarities in a study of lateral prefrontal cortex. Using a delayed matching-to-sample paradigm, RS-like sharpening of representation was observed for familiar (i.e., used in the task over many sessions) versus novel stimuli. In addition to sharpening of representation, behavioral and neurophysiological measures showed: (1) familiar stimuli were more resistant to stimulus degradation, in the sense that individual neurons tuned to familiar stimuli showed discriminatory responses at higher levels of degradation than neurons tuned to novel stimuli; and (2) neurons tuned to a familiar stimulus showed a greater selectivity of response—i.e., tendency to respond to only that stimulus— than neurons tuned to novel stimuli. Although the mechanisms of adaptation underlying these effects may be unrelated to those giving rise to RS, a parsimonious account might be able to integrate the two sets of findings. We present such an account. Existing accounts of repetition suppression are little more than descriptions of the data. Wiggs and Martin [16] and Desimone [2] suggest that cells unnecessary for identifying an item are suppressed, yielding a sparser and more selective representation, which presumably leads to a more efficient or rapid response. Ringo [13] proposes that suppression of familiar items may contribute to automatic orientation to novel items. Why is repetition suppression important? First, the robustness and ubiquity of RS suggests it may be a fundamental mechanism of adaptation in neocortex. Second, RS has become a key tool for discovering the nature of cortical representations in neuroimaging (e.g., [4, 5, 9]), based on the following argument: If representations in a cortical region are invariant to some dimension of a stimulus, then RS should be observed in that region even when the stimulus repetitions differ along that dimension. For example, in the visual word form area, RS occurs even if the two presentations differ in case, indicating a case-invariant representation; and RS is observed in higher visual areas even if the stimulus varies in retinal size or location, indicating a representation that is somewhat transformation invariant. Because of the key role RS has come to play in cognitive neuroscience, it is important to develop a theoretical perspective that supports the methodology. 1 Blind Equalization We propose an account of repetition suppression based on the innocuous assumption that a subset of cortical neurons encode intrinsically binary hypotheses. These binary-hypothesis neurons may have graded firing rates, but the firing rate indicates confidence in or probability of the truth of a hypothesis, not a continuous value (e.g., intensity or frequency). A neuron’s firing rate is characterized as the fraction of its maximal rate, yielding a value in [0, 1]. A binary-hypothesis neuron signals “false” or “true” via a value near to 0 or 1. Now consider a multilayer neural network whose outputs are binary-hypothesis neurons. The analog nature of the neural net readily allows noise to corrupt the firing rates of neurons, causing a low-confidence output (an output near .5) to flip its binary state from false to true or vice versa. For this reason, many digital communication systems that operate on underlying analog representations include an equalization process that attempts to undo the effects of noise and other distortions. If the desired output values of the neural net were known, the neural net could be trained via gradient descent to minimize the squared difference between the desired value for neuron , , and the actual value produced by the neural net, [15]. Such training removes uncertainty in the net’s output, making it more resistant to noise perturbations. Equalization is feasible even in the absence of supervision by inferring the desired value from the actual value: if or otherwise, where typically . This scheme for blind equalization depends on the assumption that the corruption of the analog firing rate is sufficiently small that its binary counterpart can be recovered, albeit at the cost of possibly losing information about confidence or probability. Thus, blind equalization trades off the ability to maintain gradations of certainty for noise robustness. If the brain can be characterized as a noisy system—whether the noise is intrinsic to neural dynamics, due to integration of conflicting cues, or due to the failure of attention to suppress irrelevant inputs—then incorporating blind equalization is a sensible, adaptive strategy. Although we focus on this mechanism of unsupervised learning, we suppose that it serves to supplement, not replace, other supervised and reinforcement learning mechanisms that operate in parallel. Blind equalization conditions representations for noise robustness, whereas supervised and reinforcement learning mechanisms achieve transformations of representations that are useful for specific cognitive activities. Therefore, blind equalization should be broadly applied to all incoming stimuli regardless of their immediate behavioral relevance—a defining characteristic of RS. Effectively, blind equalization turns up the gain of the neural response function. That is, consider a sigmoidal function relating a neuron’s summed input to its firing rate, where the gain controls the steepness of the sigmoid. Low and high gain correspond to nearly linear and more step-like response functions. Other theorists have proposed brain mechanisms that dynamically modulate the gain of response functions (e.g., [1]). However, one distinct aspect of the present proposal is that the gain modulation is linked to the specific stimulus that was presented, as well as to highly similar stimuli. Thus, one can conceive of blind equalization as setting the gain on responses to a stimulus that increases with the frequency of recent encounters with the stimulus. It is easy to get an intuition for why blind equalization leads to RS. Output neurons producing a strong response to the first stimulus presentation will increase their firing rates for a second presentation, whereas those producing a weak initial response will decrease their firing rates. Because neural codes in higher cortical areas such as IT and prefrontal regions appear to be sparse, more neurons will give a weak than strong response initially. Consequently, more neurons will decrease their activity than increase, and the overall effect is a decrease in activity. The noise robustness property described in [12] also naturally emerges from the model: Given a sigmoidal response function, noise input will have little influence on a neuron’s output if the output is close to saturation (0 or 1). 2 Simulation Methodology We model a biological neural net using the simple connectionist abstraction. Each unit in the connectionist net conveys a scalar activation level in [0, 1], interpreted as a mean firing rate relative to the unit’s maximum firing rate. We explore fully layered, feedforward nets with logistic acti