Remapping for visual stability

Nathan J. Hall, Carol L. Colby


Visual perception is based on both incoming sensory signals and information about ongoing actions. Recordings from single neurons have shown that corollary discharge signals can influence visual representations in parietal, frontal and extrastriate visual cortex, as well as the superior colliculus (SC). In each of these areas, visual representations are remapped in conjunction with eye movements. Remapping provides a mechanism for creating a stable, eye-centred map of salient locations. Temporal and spatial aspects of remapping are highly variable from cell to cell and area to area. Most neurons in the lateral intraparietal area remap stimulus traces, as do many neurons in closely allied areas such as the frontal eye fields the SC and extrastriate area V3A. Remapping is not purely a cortical phenomenon. Stimulus traces are remapped from one hemifield to the other even when direct cortico-cortical connections are removed. The neural circuitry that produces remapping is distinguished by significant plasticity, suggesting that updating of salient stimuli is fundamental for spatial stability and visuospatial behaviour. These findings provide new evidence that a unified and stable representation of visual space is constructed by redundant circuitry, comprising cortical and subcortical pathways, with a remarkable capacity for reorganization.

1. Introduction

We are usually oblivious to the changes in the retinal image that occur with each eye movement. This perceptual stability has long been understood to reflect the fact that what we see is not a direct impression of the external world but a construction, an internal representation of it. This internal representation is adjusted, or updated, in conjunction with eye movements. One neural mechanism that contributes to this adjustment is called remapping. The core idea is that visual information about salient spatial locations is maintained across saccades. Remapping is initiated by a corollary discharge of the eye movement command, which provides information about the intention to make a saccade. It does not occur if objects in the environment move but is specific to the intention to move the eyes. Remapping is limited to attended locations. It is accomplished not by a single brain area but by the participation of parietal, frontal and extrastriate cortex as well as subcortical structures. Many questions remain unanswered about the neural mechanisms that contribute to visual stability (see [1] for review). In the following sections, we will focus on three questions specific to remapping in single neurons. First, what are the spatial and temporal characteristics of remapping? Second, what neural signals and brain areas are involved? Finally, is remapping exclusively a function of cortex?

2. Temporal and spatial aspects of remapping

How does the brain keep track of salient locations when the eyes move? In parietal, frontal and extrastriate cortex, and in the superior colliculus (SC), neurons update or ‘remap’ stimulus representations in conjunction with saccadic eye movements. Updating reflects a transfer of visual information from neurons that encode a salient location before a saccade to those that encode the location after a saccade. This transfer is thought to be initiated by a copy of the oculomotor command, called a corollary discharge. Visual information is updated to reflect the effects of a saccade of a specific size and direction. The brain circuits that produce remapping integrate visual and motor signals. Remapping is a neural correlate of the brain's ability to adjust its response to visual stimuli based on internal movement commands. Remapping could thus contribute to maintaining a stable, accurate spatial representation despite the discontinuities introduced by saccadic eye movements.

Remapping of visual information can occur before, during or after a saccade [2]. In the V3A neuron shown in figure 1, remapping occurs in conjunction with the saccade [3]. The cell responds strongly to a visual stimulus presented briefly in the receptive field (RF; figure 1a). It does not respond to a stimulus presented outside the RF (figure 1c) or in conjunction with a saccade from one fixation point to another (figure 1d). The unexpected finding is that the cell does respond in the ‘single-step’ task, in which the stimulus and the saccade are combined (figure 1b). This is a visually guided saccade task that requires the monkey to make a single saccade from one fixation point to another. While the monkey fixates the first fixation point, two events occur simultaneously: a stimulus is flashed for 50 ms outside the RF of the neuron and the new fixation point appears. The location of the new fixation point is selected so that the stimulated location will be within the neuron's RF after the saccade. A burst of activity occurs (figure 1b) even though the stimulus is no longer present by the time the saccade is initiated to the new fixation point. In fact, no visual stimulus is ever present in the neuron's RF, either before or after the saccade. The stimulus appears only at the screen location where the neuron's RF will be after completion of the saccade. The control tasks (figure 1c,d) demonstrate that the remapped response requires the specific combination of a stimulus and the saccade, which will bring the stimulated location into the neuron's RF.

Figure 1.

A V3A neuron that responds to the stimulus trace after a saccade. The cartoons show the locations of the stimulus (small vertical bar) and the receptive field. Time lines in each panel show horizontal and vertical eye position for 10 trials (calibration bar 20°) and timing of task events (calibration bar 100 ms). Rasters from 10 correct trials are aligned on the events specified and summed to generate histograms. (a) Fixation task. The stimulus is flashed on for 50 ms in the receptive field (RF) while the monkey maintains fixation on the fixation point (FP). (b) Single-step task. The stimulus is flashed for 50 ms while the monkey fixates FP1. FP1 is extinguished at the same time that FP2 appears. The monkey makes a saccade to FP2. This saccade moves the RF from its original position (dashed circle) to a new position (solid circle). Note that the stimulus is extinguished before the eye reaches FP2, so that no physical stimulus ever appears in either the old or the new RF. The neuron responds to the memory trace of the stimulus. (c) Stimulus-only control. The neuron does not respond to a stimulus presented outside the RF. (d) Saccade-only control. The saccade alone does not drive the neuron. Saccade target distance, size and location of RF are drawn to scale. Adapted from Nakamura & Colby [3].

Both temporal and spatial aspects of remapping vary across neurons. In area V3A, we tested sensitivity at two locations: we measured responses to stimuli presented at the original (pre-saccade) and new (post-saccade) RF locations (figure 2). In addition, we presented the stimuli at four different times relative to the saccade. All eight conditions (two stimulus locations, four timings) were randomly interleaved. We found that changes in the RF fell along a continuum. For some, like the neuron illustrated in figures 1 and 2, the location of the RF shifted around the time of an intended saccade: they simultaneously became less responsive at the original RF location and became much more responsive at the new RF location. At time 1, well before saccade initiation, the neuron responded strongly to a stimulus flashed in the original (old) RF (figure 2a). By contrast, the response to the same stimulus in the same location was significantly reduced at time 2, when the stimulus was presented just before the saccade. It is as though the RF had already begun to shift to the spatial location that it would occupy after the saccade. When the stimulus was presented at the original RF location after the saccade had already begun there was, of course, no response (times 3 and 4). These results for stimuli presented at the original RF location are the opposite of those observed for stimuli presented in the new RF location (figure 2c). There was no response to a stimulus at the new RF when it was presented well before the saccade (time 1). The neuron did respond though to a stimulus that appeared in the new RF immediately before the saccade (time 2). The stimulus was presented for only 50 ms, thus the neuron was responding to the memory trace of a stimulus which was no longer present (note that these data for time 2 are the same rasters shown in figure 1b but are here aligned on stimulus onset). Again, as in figure 2a, it appears that the RF has shifted to the location that it will occupy after the saccade. The neuron is responsive to a stimulus that would be in the RF after the saccade. At times 3 and 4, the neuron is responding as expected to an actual stimulus in the RF, although the amplitude of response is reduced at time 3.

Figure 2.

Timing of remapping in a V3A neuron (same neuron as figure 1). The time lines (a) show when the stimulus was presented relative to the saccade. The stimulus was presented either in the old receptive field (b) or in the new receptive field (c). All eight trial types were randomly interleaved. The data are aligned on stimulus onset. The average time of the saccade is indicated by the inverted triangle above each set of rasters. The response to a stimulus in the old RF (b) is reduced when the stimulus is presented immediately before the saccade (time 2). The neuron also responds to a stimulus in the new (future) RF (c) at time 2, even though there is no physical stimulus on the screen by the time the eye reaches the new FP. The neuron is responding to the updated memory trace of the stimulus. Adapted from Nakamura & Colby [3].

The shifting RF seen for the neuron illustrated in figures 1 and 2 contrasts with the spatial and temporal pattern observed in other neurons. Some neurons appeared to become responsive at multiple locations immediately before a saccade. The neuron illustrated in figure 3 exhibited strongly predictive remapping: it began to respond well before a saccade that would move its RF onto the stimulated location (figure 3b). When tested in all eight conditions, the neuron exhibited a dual responsiveness (figure 4). At times 1 and 2, the neuron responded to a stimulus presented at either the original or new RF locations. This dual responsiveness continued until saccade onset, when it abruptly ceased (time 3). This finding suggests that there is a temporary expansion of the effective RF around the time of a saccade. The RF expansion is not necessarily uniform. It may be that only two specific locations (original and new RF) can drive the cell, so that the RF would be shaped like a barbell. Alternatively, the RF may expand uniformly and encompass locations between the original and new RF locations as well. The findings of Sommer & Wurtz [4] indicate that frontal eye field (FEF) neurons respond only at the original and new RF locations and not at locations in between.

Figure 3.

Predictive remapping in V3A. This neuron responds even before the saccade when a stimulus is flashed in the new RF. Same format as figure 1. (a) Fixation task. Visual response to 50 ms stimulus flashed in the RF. (b) Single-step task. The same stimulus is flashed for 50 ms at the future location of the RF. The neuron responds even before the beginning of the saccade that will bring the stimulated location into the RF, as though the RF had already shifted in anticipation. No physical stimulus ever appears in either the old or the new RF. (c) Stimulus-only control. The neuron does not respond to a stimulus presented outside the RF. (d) Saccade-only control. The saccade alone does not drive the neuron. Adapted from Nakamura & Colby [3].

Figure 4.

Timing of remapping for a predictive V3A neuron (same neuron as figure 3). The time lines (a) show when the stimulus was presented relative to the saccade. The stimulus was presented either in the old receptive field (b) or in the new receptive field (c). All eight trial types were randomly interleaved. The data are aligned on stimulus onset. The average time of the saccade is indicated by the inverted triangle above each set of rasters. This predictive neuron responds to a stimulus at either the old or new RF location when the stimulus is presented either long before the saccade (time 1) or immediately before the saccade (time 2). This dual sensitivity ends abruptly at the time of the saccade (times 3 and 4). Adapted from Nakamura & Colby [3].

Neurons in other cortical areas also become responsive at multiple locations around the time of a saccade. Dual responsiveness at the original and new RF locations has been observed in both lateral intraparietal area (LIP) and FEF [4,5].

The effect of such changes in RFs on visual perception is not fully understood. Changes in RFs are likely to warp visual space during saccades. Psychophysical studies in monkeys have suggested that RF shifts/expansions may underlie saccadic mislocalization [6]. Human studies are consistent with findings on remapping in monkeys (see [7] for review). For example, visual sensitivity is increased near a saccade target around the time of a saccade [8]. Multiple kinds of evidence suggest that humans and monkeys may use similar updating mechanisms for generating stable percepts. Behavioural studies indicate that both species have comparable abilities in eye movement tasks that require updating [9,10]. As discussed below, data from neuropsychological and inactivation studies suggest that the parietal lobe is crucial for these abilities in both species [1114]. Direct physiological evidence from functional magnetic resonance imaging (fMRI) studies in humans has also shown that remapping occurs in the human parietal cortex [15,16] as well as in human extrastriate cortex [17].

From personal experience we can tell that visual stability in humans is equally good following eye movements of any size or direction. Correspondingly, studies in humans have shown that double-step accuracy is equal for vertical and horizontal saccades [18]. Remapping in single neurons is likewise equivalent across different locations and eye movements. In area LIP, spatial locations are updated equally well regardless of saccade direction [19]. Strength of remapping was measured for neurons in the four cardinal directions. Although most neurons do not remap for each direction, as a population LIP neurons remap with equal strength across all four directions. In accordance, it was found that remapping is not dependent on RF eccentricity. Finally, it has been demonstrated that the amplitude of the remapped response in a given neuron in V3A is the same for 10° and 20° saccades [3]. Collectively, these studies make clear that regions which remap receive information from all locations of the visual field. This allows updating to function independently of saccade size and direction.

3. Brain signals and circuits for remapping

(a) Corollary discharge signals

Both motor and visual signals are required for remapping. The basic idea is that a corollary discharge of the eye movement command controls the transfer of visual information from neurons representing a stimulus before a saccade to those that will represent it after a saccade [20]. There are several reasons to think that remapping depends on a corollary discharge signal rather than proprioceptive feedback about eye position. First, individual neurons often exhibit remapping responses before an eye movement is initiated [2,3,5,21], that is, before proprioception could take place. Second, direct tests on proprioception have shown that it is not necessary for remapping to occur [22]. Even when all proprioceptive feedback from the eyes has been removed, behaviour that requires spatial updating remains intact. Recent ground-breaking studies by Sommer & Wurtz [4,23] have revealed a corollary discharge pathway in primates. Previous studies have extensively demonstrated corollary discharge in many smaller vertebrates [24]. Sommer & Wurtz discovered a pathway in primates that extends from the intermediate layers of SC to the FEF via the mediodorsal (MD) nucleus of the thalamus. By inactivating the MD nucleus, they showed that corollary discharge signals passing to FEF were disrupted and spatial updating was impaired [23]. They found that strength of remapping activity in FEF was also substantially reduced by inactivation of MD [4].

Because area LIP is central to remapping, it is likely that it also receives corollary discharge signals. Although specific pathways carrying corollary discharge signals to area LIP have not yet been demonstrated directly, LIP (and extrastriate cortex) could receive corollary discharge signals from FEF. Area LIP and the FEF are known to be strongly interconnected [2530]. It is possible that area LIP receives corollary discharge signals from the SC as well, via the pulvinar [31], a pathway that has not yet been physiologically investigated. Additional pathways for corollary discharge and remapping are likely to exist in the brain. Given that inactivation of MD thalamus causes only a partial deficit in double-step performance [23], additional pathways carrying corollary discharge must exist.

(b) Brain regions that exhibit remapping

Remapping was first demonstrated in the cortical area LIP [2]. Since this discovery many other brain areas have been shown to have neurons that remap, including the FEFs [32] and the SC [33,34]. These three brain areas are all strongly interconnected [29,31,3537] and contain neural signals related to both visual stimuli and saccadic eye movements [3841].

If remapping is important for visual stability, it should also exist in the ‘purely visual’ areas that are known to be involved in visual perception. Using the same tasks and conditions as in LIP and V3A, we tested neurons in V3, V2 and striate cortex [3]. Many neurons in area V3A respond in the single-step task (52% of those tested). That number drops off rapidly in V3 (35%), V2 (11%) and V1 (2%). In addition to the number of responsive neurons, the strength of response also decreases as we move down the hierarchy from V3A. Two other trends are clear. First, the proportion of neurons that remap predictively decreases markedly at lower levels of the visual hierarchy. Predictive remapping occurs in about 35 per cent of LIP neurons and 16 per cent of those in V3A. In areas V2 and V1, no neurons were found that remapped predictively. Second, there is a corresponding increase in the mean latency of the remapped response relative to saccade onset at lower levels. In humans, functional imaging studies have also demonstrated remapping in extrastriate visual cortex with a similar reduction in strength at earlier levels of the hierarchy [17]. All of these findings suggest that earlier stages of the visual system are connectionally or computationally further from the source of the central signals that drive remapping.

(c) Extrastriate circuitry for remapping

What is the neural circuit that produces remapping in extrastriate cortex? There are at least two possibilities. First, it could be that the initial computation is carried out in area LIP. A corollary discharge signal, possibly from FEF, could induce remapping of stimulus representations in LIP and the results of that computation could be fed back to earlier levels of the visual system. Numerous studies have demonstrated the strength of feedback projections from LIP to areas V3A, V3 and V2 [4246]. The large spatial scale over which remapping occurs favours the idea that the computation is carried out in area LIP: in the standard task, the saccade size is normally 20°. The transfer of visual signals over such long distances suggests that remapping is computed at a relatively high level of the visual hierarchy. Here, integration over long distances could take place. Visual RFs are much larger in area LIP than in lower order visual areas. A single LIP RF encompasses many V1 RFs. The size of LIP RFs could presumably facilitate the transfer of stimulus memory traces.

A second possibility is that remapping is produced independently at multiple levels of the visual system. Extrastriate cortex receives direct projections from FEF that could carry the corollary discharge signal needed to trigger remapping [47,48]. The density of these projections from FEF to extrastriate cortex matches quite well with the proportion of neurons in each area that exhibit remapping [30]. The idea is that corollary discharge signals from these FEF projections could trigger remapping based on local transmission of visual signals within each area and between homologous regions across hemispheres. Horizontal connections are ubiquitous in visual cortex [4951] and they serve to link more distant parts of the field at higher levels of the hierarchy [52]. New experiments are needed to constrain hypotheses about the neural circuits that give rise to remapping in extrastriate cortex.

(d) Which brain regions are critical for remapping?

Neurons in many brain regions remap, but are any particular areas required for behaviour that depends on visual stability? We might expect that areas where many cells remap are more important for spatial updating than those where it is less common. Areas that have the strongest and most abundant remapping are those that contain both visual and motor signals, such as area LIP, FEF and SC. This is logical because remapping requires a combination of visual input and information about an upcoming saccade.

In humans, both parietal and frontal cortex are important for performance in the double-step task, an eye movement task that requires remapping (see figure 5 for task diagram). Both humans and monkeys are able to perform the double-step task accurately [9,5358]. In essence, the double-step task is an extension of the single-step task: two target lights (T1 and T2) are flashed in rapid succession while the subject maintains central fixation. The instruction is simply to look at the lights in the order of appearance. Because the light flashes are brief, the targets are no longer present during the eye movements. Accurate performance in the double-step task requires information about the target location, which is provided by remapping. Programming the first saccade to T1 is straightforward. It is simply a visually guided saccade: the size and direction of the required saccade match the retinal position of the first stimulus. Programming the second saccade to T2 presents a challenge: the system must take into account the difference between the initial eye position, from which the second target was seen, and the new eye position, from which it must be acquired. Remapping the stimulus trace of the second target from the coordinates of the initial eye position to the coordinates of the new eye position accomplishes the required transformation of spatial information.

Figure 5.

Impairment and recovery of updating behaviour when direct cortical links are disrupted. Top panels show the double-step conditions used to test updating behaviour in split-brain monkeys. In the across-hemifield condition (a), the second target (T2) appears in the right visual field when the eyes are at central fixation and, therefore, is initially represented by neurons in the left hemisphere (black T2). When the eyes reach the first target T1, the memory trace of T2 is now located in the left visual field and encoded by neurons in the right hemisphere (grey T2′). Updating in this condition must involve a transfer of visual information between cortical hemispheres. In the within-hemifield condition (b), T2 is in the right visual field both at FP and at T1; updating, therefore, involves communication within the same hemisphere. Behavioural testing revealed an initial impairment on across-hemifield sequences (c). The inset shows the six randomly interleaved sequences that were tested: trained central sequences (black) and novel within (green) and across (red) sequences. Eye traces show double-step performance in the first 10 trials of the first testing session for monkeys EM (top) and CH (bottom). Dots indicate the locations of FP, T1 and T2; scale bar represents 10°. Summary data from the first session (d) show the second-saccade endpoints, which indicate a persistent impairment for monkey EM in both visual fields. There is rapid improvement for monkey CH in the left but not the right hemifield. Summary data from the final session of testing (e) show that both monkeys ultimately achieved successful performance on across-hemifield sequences. Adapted from Berman et al. [61].

In humans, there is a clear dissociation in the roles of frontal and parietal cortex. Studies in patients with unilateral parietal lesions show that accurate updating in the double-step task depends on parietal cortex [11,12,59]. When asked to perform the double-step task, these patients generate slow double-step saccade sequences. They are only able to perform the two saccades accurately when both targets remain visible. If T2 is extinguished before the first saccade, so that rapid spatial updating is required, these patients are unable to complete the task. The impairment is not simply owing to visual or attentional deficits. Saccades toward T2 appear to be random, even though the correct saccade direction and the initial location of T2 were presented to the intact hemisphere. The deficit appears to reflect an inability to update the new retinal location of T2 relative to T1. These observations suggest that patients with parietal damage experience deficits that are specific to updating even though motor planning remains intact. This finding is consistent with results from inactivation studies in monkey area LIP. These animals show increased latency and decreased accuracy during saccades to T2 [13].

Patients with frontal lobe damage exhibit different symptoms. These patients appear to have general motor impairments and perform poorly on the double-step task even when T2 remains visible [12]. Their saccades are both slower and less accurate than normal. On the other hand, saccades to T2 are oriented in the correct direction, as though remapping were intact. Studies on parietal and frontal lobe patients thus point to different roles for the two brain regions. Parietal cortex seems to be necessary for accurate remapping but not necessary for visually guided saccade control. Frontal cortex plays a more prominent role in the motor aspects of the double-step task. These results in humans are consistent with findings from frontal lesion studies in the monkey [60]. Performance on a task similar to the double-step task remains intact after lesions of the FEF. The conclusion in both humans and monkeys is that parietal cortex is necessary for accurate spatial updating in the double-step task while frontal cortex is not.

4. Cortical and subcortical contributions to remapping

Our central idea is that spatial updating involves an active transfer of information between two sets of neurons. Previous studies have demonstrated that this transfer can take place over long spatial distances, even across visual hemifields. Remapping across visual hemifields presumably requires communication between sets of neurons in opposite cerebral hemispheres. What circuitry produces this interhemispheric transfer of information? We hypothesized that the corpus callosum provides the primary route for updating salient spatial locations across visual hemifields.

We tested this hypothesis in two split-brain monkeys by measuring performance in the double-step task. In split-brain monkeys, the two cortical hemispheres are completely disconnected by removal of the forebrain commissures—the entire corpus callosum and the anterior commissure. We predicted that removal of the forebrain commissures would eliminate the ability to remap stimulus traces from one hemifield to the other. We measured the performance of split-brain and intact monkeys in the double-step saccade task [61,62]. We compared two kinds of sequences: across-hemifield sequences, in which T2 was updated from one visual hemifield to the other (figure 5a), and within-hemifield sequences, in which T2 was updated within the same hemifield (figure 5b). At the beginning of testing, these novel sequences were interleaved with a well-learned central sequence in which T2 was on the vertical meridian. Each monkey was initially trained on a set of within-hemifield sequences (the first saccade was always vertical). In these conditions, successful performance could be achieved through within-hemifield updating regardless of the direction of the second saccade. After the initial phase of training, a set of novel conditions was introduced in which the initial saccade was horizontal and T2 could appear at any of six symmetrically placed locations in either the right or the left visual field. Depending on the location of T2, successful performance depended on either within-hemifield or across-hemifield updating. Each monkey initially failed completely on across-hemifield conditions while performing accurately on within-hemifield conditions. Figure 5c shows performance on the first 10 trials for each monkey. Both animals were very accurate on the trained central sequences, and performed well on the new within-hemifield conditions: saccade endpoints for the central (black) and within (green) sequences are clustered near the correct T2 locations. In contrast, the monkeys missed every trial of the first 10 across-hemifield sequences (red). As expected, the intact animal was able to perform the novel across-hemifield sequences without difficulty (not shown). These data are consistent with the prediction that across-hemifield performance would be impaired in the absence of the forebrain commissures.

To our surprise, both animals learned to perform the across-hemifield double-step as they gained experience with specific sequences. Even during the first day of testing, performance improved markedly (figure 5d). Performance on across-hemifield conditions over dozens of daily sessions ultimately improved so that it became nearly as good as within-hemifield performance (figure 5e). The impact of experience was location-specific: changing the geometry of the targets resulted in a reinstated across-hemifield deficit that was ameliorated only after another phase of training. This acquired updating ability was systematic, not just categorical: small trial-to-trial variations in the location of T2 elicited matching small variations in the endpoint of the second saccade. This stimulus-dependence means that the animal was actually using updated visual information, rather than relying on a motor strategy. We conclude that, while the forebrain commissures provide the primary path for updating spatial information across hemifields, subcortical pathways must be responsible for the recovery of function. The circuitry that underlies remapping is both plastic and redundant: other pathways are capable of taking over the transfer of remapped signals in split-brain animals.

(a) Does cortex still have a role in remapping in split-brain animals?

These results led us to ask what role cortex had in recovered remapping circuitry in the split-brain monkeys. To answer this question, we investigated neural activity in area LIP of the same split-brain monkeys during the single-step task [21]. If subcortical structures and pathways had taken over, it was possible that there would be no physiological evidence of remapping in cortex. Instead, we found that neurons in LIP still exhibit remapping in the across-hemifield condition. Activity during across-hemifield trials was reduced when compared with activity during within-hemifield trials. Also, the latency of remapping signals is selectively increased during across-hemifield trials. Overall, remapping signals are weaker and take longer to be transferred in the across-hemifield condition in split-brain animals. However, LIP still receives remapping information and thus could remain an important structure for spatial updating.

The role of the forebrain commissures in remapping has been shown to be primarily visual [63]. Split-brain monkeys were tested on a modified version of the double-step task (the ‘motor-across task’) that dissociates interhemispheric transfer of corollary discharge signals from the transfer of visual signals. The critical modification is that T2 is located in the visual hemifield opposite T1 when testing the transfer of corollary discharge. So, for example, if T1 appears in the left visual field, T2 would appear in the right visual field. To perform the task, the monkey makes an initial, leftward saccade to T1 triggered by neurons in the right cortical hemisphere. However, because T2 is located in the right visual field, it is represented by visual neurons in the left hemisphere. In order for visual neurons to update the location of T2 with respect to T1, they must receive a corollary discharge signal from the motor neurons in the right (opposite) cortical hemisphere that initiated the saccade to T1. In contrast to their performance on the original double-step task, which tests transfer of visual signals, the split-brain monkeys were not impaired on this modified version of the task. The conclusion is that subcortical structures that transfer corollary discharge signals do not rely on the forebrain commissures for interhemispheric transfer. This finding is understandable in light of the fundamental discovery that corollary discharge signals are sent from SC to FEF [23]. The SC projects to FEF both ipsilaterally [23,64] and contralaterally [65]. The same arrangement has been shown in projections from SC to LIP [31]. The net result of the split-brain monkey studies is that cortical areas implicated in remapping still receive the critical visual and motor signals required for this process—even if the forebrain commissures are removed.

(b) Subcortical remapping

The behavioural results described above indicate that spatial updating is not dependent on direct cortico-cortical connections. The improvement in performance seen in split-brain monkeys on across-hemifield trials tells us that alternative circuitry exists that can carry remapping signals across hemispheres. Because interhemispheric connections in subcortical structures are unaffected in the split-brain monkeys, it is likely that subcortical structure(s) are involved. A prime candidate for inclusion in this circuit is the SC. The SC is a multi-layered structure. Neurons in the superficial layers are primarily visual. Neurons in the intermediate layers have both visual and saccade-related activity [38]. Neurons in the intermediate layers also exhibit remapping [34]. Connectionally, the SC is very well suited to contribute to remapping. It has extensive connections with many visual and visuomotor cortical areas [64,6671], including area LIP [31,35,7276]. To explore the role of collicular circuits in remapping, we analysed the activity of SC neurons during the double- and single-step tasks in split-brain monkeys.

Distinct patterns of activity were found in each SC layer [33]. Remapping activity for across-hemifield conditions was diminished in the intermediate layers of split-brain monkeys (figure 6c). Both the magnitude of the response and the number of cells that showed remapping in across trials were reduced when compared to within trials. The latency of the neural remapping response was also longer for across-hemifield trials. These results mirror those found in area LIP of split-brain monkeys [61]. In contrast, remapping magnitude (figure 6a) and latency were equal in neurons of the superficial layers, with strictly visual responses, in split-brain monkeys. These findings were specific to split-brain animals. The intact animal showed equal remapping within- and across-hemifields in both layers of the SC (figure 6b,d).

Figure 6.

Within–across (WA) index distributions in superficial and intermediate SC for split-brain and intact monkeys. Indices are calculated as: [(neural activity for within trials)−(neural activity for across trials)]/(sum of activity on within and across trials). Values are normalized to a saccade control condition before calculation. Values were calculated for all neurons with significant remapping in both within and across conditions. Positive WA indicates stronger remapping during within conditions. WA index distribution for superficial layers is not significantly biased towards within or across conditions in either split-brain (a) or intact monkeys (b). Split-brain monkeys have significantly more cells with positive WA indices in the intermediate layers (c). The intact monkey shows no significant shift in WA index values (d). From Dunn et al. [33].

The striking differences between remapping in superficial and intermediate SC layers indicate that they may be part of different functional circuits. The relative impairments in intermediate layer neural activity suggest that remapping here is at least partly dependent on cortical structures. Area LIP (or other cortical areas, such as FEFs) may be an important source of remapping for these SC cells. Although this novel finding is of interest in itself, it also tells us that remapping in intermediate SC is unlikely to contribute to the recovered spatial updating observed behaviourally in split-brain animals. Nor is it likely to contribute to the remaining across-hemifield remapping response in LIP. Instead it suggests that neurons in the intermediate layers may not be able to remap without cortical input: any brain region underlying recovered remapping would be expected to be unaffected by cortical disconnection.

Remapping in superficial layers during across-hemifield trials is robust in the absence of the forebrain commissures. The magnitude and latency of remapping are unaffected by cortical disconnection in these layers and do not appear to mirror LIP activity. Thus, remapping activity in the superficial layers of SC is not dependent on cortical input. Instead, superficial SC neurons must either receive remapped information from another subcortical structure, or they are able to create remapped responses on their own. This observation may provide a clue as to how the split-brain monkeys recovered spatial updating capabilities in the across-hemifield double-step task. It demonstrates that multiple neural pathways exist that are capable of carrying out remapping.

5. Conclusions

In this review, we explored three main questions. First, what are the spatial and temporal characteristics of remapping? Remapping of briefly flashed stimuli can occur during or after a saccade and can even occur before a saccade is initiated. RFs can undergo substantial spatial changes during remapping, with some neurons becoming responsive at two locations simultaneously.

Our second question concerned which brain areas are involved in remapping. Many neurons in oculomotor areas, such as the FEFs and SC, exhibit remapping. Remapping is most common in neurons in area LIP, in keeping with the primary role of parietal cortex in spatial attention. Remapping is also prevalent in neurons throughout the dorsal stream. This is important because it indicates that remapping is a general phenomenon, not limited to oculomotor and attentional structures. Frontal and parietal cortices appear to have complementary roles in remapping. Patients with lesions in frontal or parietal cortex show different kinds of deficits in the double-step task. Parietal cortex seems to be essential for spatial representation while frontal cortex is more important for motor control.

Finally, we asked whether remapping is exclusively a function of cortex. We found that after transection of the forebrain commissures the behaviour of split-brain monkeys in the double-step task ultimately recovers. Neurons in area LIP (and possibly other cortical areas) exhibit remapping even after the hemispheres have been disconnected. Both the superficial and the intermediate layers of the SC participate in remapping. The SC, and perhaps other unexplored subcortical pathways, may contribute to recovered double-step behaviour and visual stability. These studies show clearly that remapping is a complex and critical cognitive function. Remapping is pervasive throughout the brain, both cortically and subcortically, and shows powerful plasticity and redundancy when its primary systems are damaged.



View Abstract