Recordings of rat hippocampal place cells have provided information about how the hippocampus retrieves memory sequences. One line of evidence has to do with phase precession, a process organized by theta and gamma oscillations. This precession can be interpreted as the cued prediction of the sequence of upcoming positions. In support of this interpretation, experiments in two-dimensional environments and on a cue-rich linear track demonstrate that many cells represent a position ahead of the animal and that this position is the same irrespective of which direction the rat is coming from. Other lines of investigation have demonstrated that such predictive processes also occur in the non-spatial domain and that retrieval can be internally or externally cued. The mechanism of sequence retrieval and the usefulness of this retrieval to guide behaviour are discussed.
The theme of this special issue is the topic of prediction. There is now strong evidence that the activity in the rodent hippocampus can reflect predictive information about potential future events and places, particularly of the order of several seconds in the future. Moreover, while animals can predict outcomes in simple conditioning tasks without a hippocampus (Corbit & Balleine 2000), the hippocampus is necessary for the predictions that involve novel sequences or temporal gaps (O'Keefe & Nadel 1978; Dusek & Eichenbaum 1998; Redish 1999), particularly when this prediction requires the integration of spatially and temporally separated information (Cohen & Eichenbaum 1993; Redish 1999). In this paper, we will review the evidence that hippocampal representations show predictive sequences during behaviours that depend on hippocampal integrity.
In a general sense, any memory could contribute to the ability to form predictions, but the memory of sequences has special usefulness. Suppose that the sequence ABC has occurred in the past. The subsequent appearance of A can serve as a cue for the recall of this sequence. To the extent that a sequence that has been observed once will tend to recur, the cued recall of BC is a prediction that BC is likely to happen next.
Research on the role of the hippocampus in memory sequences has progressed rapidly over the last decade owing to three major developments. First, it has now become standard to monitor a large number of neurons in awake, behaving rats (Wilson & McNaughton 1993; Buzsaki 2004). Second, new developments in analytical methods have enabled the study of how neural ensembles represent space on a fast time scale, thereby eliminating the need to average over long, potentially variable time frames (Brown et al. 1998; Zhang et al. 1998; Johnson et al. 2008, 2009). Third, it is now clear that the hippocampus cannot be understood solely in terms of a simple rate code. Rather, important additional information is encoded by a temporal code in which cellular firing is organized by oscillations in the theta (7–10 Hz) and gamma (40–100 Hz) range.
We will begin by describing these oscillations and the way they organize information. We will then review the experimental evidence that the hippocampus retrieves memory sequences and can use this information to guide behaviour. We then turn to open questions, such as the mechanism of sequence retrieval, how far in the future the hippocampus can predict and whether there is a ‘constructive’ form of prediction about situations that have not previously occurred.
2. Theta/gamma phase code and phase precession
The oscillations of the hippocampus are easily observed in the local field potential of the rat. These oscillations depend strongly on behavioural state (Vanderwolf 1971; O'Keefe & Nadel 1978; Buzsaki 2006). During movement, the local field potential is primarily characterized by strong oscillations in the theta frequency range (7–10 Hz). At the same time, there is also a faster oscillation in the gamma frequency range (40–100 Hz; Bragin et al. 1995). These dual oscillations are shown schematically in figure 1. By contrast, during slow-wave sleep and inattentive rest, the local field potential is characterized by a broader frequency distribution and much less prominent theta and gamma. This state (termed large-amplitude irregular activity (LIA)) is punctuated by brief events called sharp waves. Although pauses characterized by inattention (eating, grooming, resting) tend to show LIA, pauses characterized by active attention (e.g. during anxiety, fear, decision making) show continued theta (Vanderwolf 1971; O'Keefe & Nadel 1978; Gray & McNaughton 2000; Johnson & Redish 2007). As we will see below, certain types of memory processes only occur during periods of theta oscillations.
When one uses firing rate as an indicator of cell activity, that activity defines the place field of the place cell. This field is generally a relatively small fraction of the environment (middle panel, figure 3g). The place fields of different cells occur in different locations; thus, the place cells collectively code for the animal's position within an environment. Two lines of evidence, however, indicate that this description is incomplete. Place cells sometimes fire spikes outside their main place fields, for example, at feeder sites (O'Keefe & Nadel 1978; Jensen & Lisman 2000; Jackson et al. 2006) and at decision points (Johnson & Redish 2007). As will be discussed later, this additional firing (extra-field firing) may actually reflect information processing (‘thinking’) about a different location—that of the cell's main place field rather than the animal's current location. The second line of evidence relates to the aspects of neural coding that go beyond what can be accounted for by a rate code. As a rat crosses the place field of a cell, the firing shows temporal coding: spikes fire with a systematic timing (phase) relationship to ongoing theta oscillations. Moreover, this relationship changes as the rat moves through the place field, a phenomenon known as ‘phase precession’ (O'Keefe & Recce 1993; Skaggs et al. 1996; Dragoi & Buzsaki 2006; Maurer & McNaughton 2007). This phenomenon is shown in figure 2a and illustrated schematically in figure 1. If a rat moves through a place field at average velocity, firing occurs during approximately 8–12 theta cycles (i.e. a total duration of approx. 1–2 s; Maurer & McNaughton 2007). Phase precession is most clearly seen on linear tracks as the rat runs through the place field. The cell initially fires during late phases of the theta cycle, but on each successive theta cycle, firing occurs at an earlier and earlier phase. Models of phase precession have suggested that phase precession contains both retrospective and prospective (predictive) components. As we will review later, it has now been definitively shown that the phase precession has a predictive component.
An issue that determines the information capacity of a phase code is how finely the phase of theta can be divided. Recent experimental work strongly supports the theoretical suggestion that the coding scheme used by the hippocampus is a theta/gamma discrete phase code (Lisman & Idiart 1995; Lisman 2005). According to this hypothesis, the gamma oscillations that occur during a theta cycle themselves control firing (i.e. firing can occur at only a restricted phase of a gamma cycle). Thus, there will be approximately 5–14 gamma cycles in each theta cycle (7/40–100 Hz) and this will result in a corresponding number of discrete times at which firing may occur. Consistent with this hypothesis, recent work (figure 2b) has found that the firing of hippocampal pyramidal cells is gamma modulated while the theta phase precession is occurring (Senior et al. 2008). The cells that fire during a given gamma cycle define an ensemble (i.e. a spatial code for a memory item). Overall, then, approximately 5–14 items can be coded for during a theta cycle, each in a given gamma subcycle of a theta cycle. This coding scheme provides a framework for understanding the phase precession in the hippocampus (see below), but may also be related to the capacity limits of working memory networks in cortex (Lisman & Idiart 1995).
3. Prospective coding: predicting upcoming places
Soon after phase precession was discovered, it was suggested that the phenomenon implies a process in which the represented location sweeps across positions in the direction of travel (Skaggs et al. 1996). Two early models specifically proposed that phase precession should be interpreted as a sweep ahead of the animal (Jensen & Lisman 1996; Tsodyks et al. 1996).
An important aspect of this interpretation (figure 1) is that the spatial resolution of the system is much finer than the size of the place field: the ‘true place field’ (which we designate here by a letter) is taken to be approximately one-seventh the size of the apparent place field (the entire field where rate is elevated). Thus, we can think of positions A, B, C, D, E, F and G as seven subparts of an apparent place field (with different cells representing each position and firing in different gamma cycles of the theta cycle). As will be explained in the following paragraphs, the firing of the ‘G-cell’ illustrated in figure 1 at positions A–F is actually a prediction about the distance to the true place field, G.
To further clarify this interpretation, let us first consider what happens during a single theta cycle, a phenomenon we term a ‘sweep’. Suppose that one is recording from a cell representing position G ahead of where the rat is now (position A). It might at first seem that this cell should not fire until the animal gets to G, however, in the context of a predictive process based on phase coding, firing of the G-cell at position A at late theta phases can be understood as a prediction that the rat is approaching G. The process that causes the G-cell to fire when the animal is at position A can be understood mechanistically as a chaining process that occurs within a theta cycle: if the rat is at position A, the cells representing position A are cued to fire at the beginning of the theta cycle (i.e. in the first gamma cycle). The chaining process then fires the next cells in the spatial sequence (these represent position B and fire in the second gamma cycle). This chaining process continues until the G-cell fires at the end of the theta cycle (in the last gamma cycle within the theta cycle). There is thus a sweep of activity from location A to location G during this theta cycle.
A key to understanding the phase precession is that the cue will change over time as the animal runs; this will change the character of sweeps in successive theta cycles. Thus, when the second theta cycle occurs, the rat will have proceeded farther along and so is now at B. Given that B is now the cue, the sweep that occurs in the second theta cycle will be BCDEFGH. It can be seen that whereas in the first theta cycle, the G-cell fired in the last gamma cycle, in the second theta cycle, the G-cell now fires in the next to last gamma cycle, i.e. with earlier theta phase. It thus follows that as long as the rat keeps moving, the G-cell will fire with earlier and earlier phase on each successive theta cycle.
4. Evidence for prospective coding in the hippocampus
In support of this interpretation, several experiments show that many spikes fired by place cells actually represent a position ahead of the animal. First, studies of phase precession in cells with omnidirectional place fields have shown that firing during the late components of theta (early firing in a pass through the place field, i.e. the predictive components) fire as the animal approaches a specific location from any direction. That is, as the animal approaches the position from the left, these spikes are fired to the left of the point; as the animal approaches the position from the right, these spikes are fired to the right of that point. This has been seen (figure 3) on both the cue-rich one-dimensional linear track (Battaglia et al. 2004) and in the standard two-dimensional cylinder foraging task (Huxter et al. 2008). These experiments suggests that the information about position implicated by firing of the cell (the true place field of the cell) is actually a small central point corresponding to where the rat is during the firing at early phases of the theta cycle (the central portion of the classic omnidirectional place field in two-dimensional, the later portion of the unidirectional place field in one-dimensional).
Second, Johnson & Redish (2007) found that when a rat comes to a difficult choice point on a maze and shows evidence of searching behaviour (the animal alternately looks left and right, a behaviour termed vicarious trial and error (VTE); Meunzinger 1938; Tolman 1939), neural ensembles within the hippocampus encode future positions ahead of the animal (figure 4). During these behavioural pauses, the animal appears attentive to its surroundings and the hippocampal local field potential remains in the theta state (Vanderwolf 1971; O'Keefe & Nadel 1978; Johnson & Redish 2007). Decoding represented positions from hippocampal ensembles showed that there are sweeps of firing that represent successive positions along one or the other arms of the maze. These sweeps show directionality away from the choice point (figure 4). These sweeps can be interpreted analogously to the prospective cued-chaining interpretation of phase precession reviewed in the preceding section. In the cued-chaining interpretation, the observed phase precession is a series of sweeps (ABCDEFG, BCDEFGH, etc.) in which each sweep is a predictive process that differs from the preceding one because of a changing cue. The sweeps observed by Johnson and Redish may be single instances of one of these chained sequences.
Finally, on tasks with highly repeated paths (such as running back and forth on a linear track, or running around a circular track), place fields expand backwards along the direction of travel, such that on later laps within a day, the place field gains a leader region in front of the place field that was seen on the first laps. This is termed place field expansion (Blum & Abbott 1996; Mehta et al. 1997). It has now been established that place field expansion does not cause phase precession—place field expansion can be inhibited, while leaving the basic phase precession intact (Shen et al. 1997; Ekstrom et al. 2001). The effect of place field expansion is to lengthen the first part of the phase precession, which can be viewed as the learned ability to predict yet further ahead in each theta cycle (Blum & Abbott 1996; Mehta et al. 1997; Redish & Touretzky 1998; Redish 1999; Jensen & Lisman 2005). This means that as the animal repeatedly observes regular sequences, the sweeps occurring during each theta cycle reach further into the future (ABCDEFGHIJ instead of just ABCDEFG), allowing an earlier prediction of approach to a location.
5. Manipulating the spatial cue for sequence recall
The interpretation of the phase precession as reflecting a cued sequence recall process depends on the assumption that the retrieval process is cued on each theta cycle by the current position of the rat (figure 1). It follows that the phase precession should depend on the velocity of the rat. Suppose, for instance, that the rat ran so fast that by the second theta cycle it was already at G. In this case, proper prediction would imply that firing should occur only on two theta cycles and the entire change in phase (from late to early) should occur in two theta cycles. Consistent with this, systematic study of the velocity dependence of phase precession indicates that phase precession is more accurately described by spatial rather than temporal traversal (O'Keefe & Recce 1993; Skaggs et al. 1996; Geisler et al. 2007; Maurer & McNaughton 2007). At average velocity, the phase advance is equivalent to 0.5–1 gamma period per theta cycle; when the rat runs slower, the phase advance is less and when the rat runs faster, the phase advance is greater.
A further test of the importance of changing cues is to put the rat in a running wheel. If the phase precession is due to internal dynamics, to proprioceptive feedback from running, or even from some general motivational or speed-of-travel signal, phase precession should still occur. However, if phase precession is primarily driven by an updating of current position cues, then phase precession should be abolished. Recordings from Buzsaki's laboratory (Czurko et al. 1999; Hirase et al. 1999) found that under simple conditions, phase precession vanished on the running wheel—instead, individual cells fired at a set phase of theta.
6. Phase precession in the non-spatial domain
The ease with which the spatial location of the rat can be studied can lead to the impression that the rat hippocampus is uniquely processing spatial information rather than being a general purpose memory device. However, lesion experiments in rats indicate that the hippocampus is also necessary for the memory of non-spatial (odour) sequences (Fortin et al. 2002; Manns et al. 2007, but see also Dudchenko et al. 2000). This helps to bring together the rat literature with the human one, where the hippocampus clearly has a role in many non-spatial aspects of episodic memory (reviewed in Cohen & Eichenbaum 1993).
Two recent studies have demonstrated phase precession in the rat in the non-spatial domain. In the first of these studies (Lenck-Santini et al. 2008), the rat was removed from a ledge and dropped (see also Lin et al. (2005) for a non-predictive comparison). Within a certain time, the rat had to jump back on the ledge to avoid a shock. Theta oscillations and phase precession were observed after the rat was picked up (just before being dropped) and again in the short period just before the rat jumped to avoid the shock (figure 5). This non-spatial phase precession can be interpreted as a prediction of being dropped and a prediction of jumping. An alternative interpretation of the first finding that the rat was predicting where it was going to be dropped was rejected because the same firing occurred irrespective of where in the environment the rat was dropped.
In the second of these studies (Pastalkova et al. 2008), a rat had to run on a running wheel for a short period during the delay period of a working memory task. When the rat was allowed to leave the running wheel, it could complete its path to a reward site that alternated between trials. During the brief period on the running wheel, phase precession occurred (compare the previous finding that no phase precession occurs if the rat is simply running indefinitely on the wheel). The phase precession in these experiments can be interpreted as using time as a cue (or, equivalently, as traversing a sequence of internally generated states; Levy 1996; Levy et al. 2005) and what is being predicted is a future event (i.e. being dropped, jumping, getting off the wheel). According to this interpretation, on each successive theta cycle, the (internal) temporal cue is advanced (just as the spatial cue is advanced during running on the linear track); thus, the chaining process on each successive theta cycle leads to earlier and earlier theta phase firing of the cells representing the event.
7. Usefulness of sequence retrieval to guide behaviour
There is now begining to be evidence that the predictions that occur during sweeps are actually used by the animal to guide behaviour. Johnson & Redish (2007) found that the direction of the sweep was strongly correlated with the direction of the animal's motion, but was not necessarily correlated with the final decision made. Johnson and Redish also found that these sweeps occurred at decision points and during error correction when animals were performing VTE-like behaviours. These behaviours are known to be hippocampally dependent (Hu & Amsel 1995), related to hippocampal activity as measured by c-fos (Hu et al. 2006) and necessary for proper decision making (Meunzinger 1938; Tolman 1939). The single sweeps observed by Johnson and Redish generally occurred during a single theta cycle and reply positions at a velocity approximately 4–15 times faster than during the actual traversal of these paths. These sweeps are thus exactly what one would predict if the animal had to rapidly recall its experience down the two arms of the maze in order to make the correct decision about which way to turn.
Additional evidence for a behavioural function of sweeps is provided by the experiments of Lenck-Santini et al. (2008), who examined hippocampal activity in the moments before the rat jumped out of the chamber to avoid a predictable shock. In such internally timed tasks, there is a strong build-up of theta activity before the animal acts (Vanderwolf 1971; O'Keefe & Nadel 1978; Terrazas et al. 1997). If this build-up of theta was disrupted, the animal often did not act. It is during this build-up that the phase precession occurred. Lenck-Santini et al. found that a well-trained rat injected with the cholinergic antagonist scopolamine sometimes failed to jump to avoid the shock. There was a strong correlation between these instances and the failure to see the build-up of theta power before the jump. Interruption of theta (and the concomitant phase precession) may have interfered with the planning of the jump.
8. Possible mechanisms of phase precession
There is now substantial evidence that sweeps (and phase precession) can be viewed as a predictive process, but the underlying mechanisms are less certain. Several models have been proposed. One class of models is the dual-oscillator model (O'Keefe & Recce 1993; Burgess et al. 2007; Hasselmo et al. 2007); we refer readers to an excellent critique of such models (Mauer & McNaughton 2007). Here, we focus on cued-chaining models.
In cued-chaining (Jensen & Lisman 1996; Tsodyks et al. 1996; Maurer & McNaughton 2007), asymmetric weights exist between the cells representing subsequent positions along the track (e.g. the synapses of A cells onto B cells, the synapses of B cells onto C cells). Thus, if after the asymmetric weights are formed, the animal subsequently comes to A, the A cells can retrieve the B–G sequence by a simple excitatory chaining process that uses these asymmetric weights. This sequence will occur during a theta cycle, thereby producing a sweep. Importantly, the processes postulated are not linked to the spatial aspects of place cells and so can be generalized to all forms of memory.
The proposition that the asymmetric weights thought to underlie the phase are learned through experience makes a strong prediction: phase precession should not occur on the first pass through a novel environment. Experimental tests of this prediction have given a somewhat mixed answer. Rosenzweig et al. (2000) reported that cells could show phase precession even on the first lap on a novel track, but this study did not attempt to quantify phase precession. In the most systematic study to date, Cheng & Frank (2008) reported that phase precession is weak on first exposure to a novel track, but rapidly becomes stronger with further experience. These results suggest that there are both learning-dependent and -independent components of the phase precession. One possibility is that the learning-independent component could be due to intrinsic single-cell biophysics (Kamondi et al. 1998; Magee 2001; Harris et al. 2002) either in the hippocampus (O'Keefe & Recce 1993) or in the entorhinal cortex (Burgess et al. 2007; Hasselmo et al. 2007) and may be related to non-predictive components (such as would occur in the dual-oscillator model; Maurer & McNaughton 2007).
Another potential mechanism of the learning-independent components of phase precession is to hypothesize that there are pre-wired directionally dependent asymmetric weights within the system (Samsonovich & McNaughton 1997). However, the connection matrix needed to implement a directionally dependent weight matrix in hippocampal place cells is very complicated owing to the remapping properties seen therein (Redish & Touretzky 1997; Maurer & McNaughton 2007). Because entorhinal cells do not remap between environments (Quirk et al. 1992; Fyhn et al. 2007), a pre-wired map could exist in the entorhinal cortex (Redish & Touretzky 1997; Redish 1999) encoded by the grid cells therein (Hafting et al. 2005). The existence of directional information within the entorhinal cortex (Hafting et al. 2008), in conjunction with a pre-existing map, could allow prediction of future locations without the animal having ever experienced the path between these locations.
The above considerations suggest that there is both a learning-dependent and a learning-independent component of the phase precession and that there are plausible mechanisms that could underlie these components. Thus, the general idea that the phase procession occurs by a chaining process that uses thase mechanisms seems plausible. However, there is an objection to this class of models that was emphasised in a recent review (Maurer & McNaughton 2007) which we will now discuss. In many environments place fields have omnidirectional fields. In these environments, place fields have omnidirectional fields. In these environments, the learning processes that drive synaptic learning should produce symmetric weights (Muller et al. 1991; Redish & Touretzky 1998). Symmetric weights should lead to excitation of both A and C when an animal is at the intermediate location B, contrary to the data showing that phase precession and sweeps produce unidirectional sequences (i.e. the sequence is either ABCD or DCBA depending on which way the rat is moving). One potential resolution to this problem would be to assume that there is a small directional component that causes cells to fire in one direction but not the other. Directionally dependent inhibitory interneurons have been found in the hippocampus (Leutgeb et al. 2000), which could selectively inhibit activity behind the animal. Alternatively, visual cues coming from the lateral entorhinal cortex (Hargreaves et al. 2005; Leutgeb et al. 2008), which would depend on which direction the rat is headed, may change the firing rates sufficiently to break the symmetry.
9. Simple retrieval or construction?
We have reviewed the evidence that the process of sequence recall can be observed in the hippocampus and that several variants can now be studied in detail. As we have argued, simple recall of sequences can be viewed as a form of prediction. To the extent that the world is governed by fixed sequences, cued sequence recall is a prediction of what will happen next. However, recall may not simply be a replay of past sequences. As noted at the beginning of this review, tasks that depend on hippocampal integrity tend to be those tasks that require integration of separate components, while simple sequence recall tends to be hippocampally independent. For example, the hippocampus is not involved in the generation of simple expectancies, as used in typical instrumental learning tasks (Corbit & Balleine 2000), but it is involved in accommodating complex changes in contingencies (as in contingency degradation tasks; Corbit et al. 2002). This ability to deal with complexity might allow the hippocampus to combine information to produce a prediction of events that never happened. From a cognitive perspective, it seems clear that both simple recall of memories and constructive processes take place. What is much less clear is where these processes take place and the particular role of the hippocampus. Perhaps the hippocampus is best described as a simple memory device: the predictive processes that occur during cued recall are a relatively faithful replay of the original events and these events are then integrated by other brain regions to construct predictions. Alternatively, the hippocampus may itself integrate information to form constructions. A recent study of patients with hippocampal damage (Hassabis et al. 2007) suggests that the role of the hippocampus is in the integration of separate components rather than the simple retrieval of memories. At this point, the electrophysiological evidence only provides support for retrieval. Whether the hippocampus can also construct never-experienced predictions is still an open question.
10. Time scales
An important limitation of the predictive process described thus far is that is it deals with a rather small temporal and spatial scale. The phase precession and sweeps discussed above are predictions about locations less than a metre from the current position and that the rat will typically come to in several seconds. Human cognitive abilities depend on the ability to predict much further in the future. Whether rats have this ability is controversial (Roberts et al. 2008).
Several mechanisms could potentially provide prediction further in the future. The experiments described in this paper were all carried out in the dorsal hippocampus, but cells in the ventral entorhinal cortex and hippocampus have much larger fields (Jung et al. 1994; Kjelstrup et al. 2008). Sweeps in these more ventral aspects could thus proceed much further in the future than those in dorsal aspects (Maurer & McNaughton 2007). Another possibility relates to the temporal scale of the hippocampal inputs through chunking (Miller 1956; Newell 1990) or the action of a working memory buffer (Jensen & Lisman 2005). It is possible that the information in each successive gamma cycle (figure 1) could represent events separated by large times. A final possibility is simply that the information at the end of a sweep could be looped back to provide the cue for another sweep, thereby extending the range of temporal associations.
The ability to electrophysiologically monitor ensembles in the hippocampus provides a way of addressing many of the open issues raised in this review. It should now be possible to study the time scales of prediction, analyse the mechanisms involved and determine the degree to which the process can be viewed as simple retrieval as opposed to construction. These questions will not only be important in their own right, but also provide a starting point for understanding how other brain regions interact with the hippocampus, e.g. by providing cues that stimulate hippocampal prediction or by using the hippocampal output to guide behaviour.
We thank Adam Johnson, Ed Richard and Matthijs Van der Meer for their helpful comments on an earlier version of the manuscript. A.D.R. was supported by MH080318. J.L. was supported by R01 NS027337 and P50 MH060450.
One contribution of 18 to a Theme Issue ‘Predictions in the brain: using our past to prepare for the future’.
- © 2009 The Royal Society