| The role of sleep in modulating representational change | 10.15154/1520667 | The aim of this study is to test the NMPH model using a new intervention – sleep – in a statistical learning paradigm that has previously been shown to drive representational change (Kim et al., 2017). In this paradigm, a small number of exposures to an item in a particular temporal context (e.g., B follows A) creates an expectation that when A appears, B should follow. If this expectation is violated (A is followed by X), the memory for mispredicted item B is activated, causing it to weaken (i.e., be forgotten; Kim et al., 2014). However, if the mispredicted item is later restudied, this interleaved sequence of trials leads to differentiation of the A and B memory representations in the hippocampus compared to a nonviolation control condition (Kim et al., 2017). This study also used fMRI pattern similarity analysis to show that moderate (vs. weak) activation of B during violation trials led to more differentiation, in keeping with the NMPH. Interestingly, this differentiation effect was not immediately apparent, and was only observed when learning and the post-learning measurement of representational change occurred on separate days with an intervening night of sleep (unpublished observation). The key question we aim to address in this study is whether interleaved learning during wake is sufficient to cause differentiation, or whether sleep (in particular, REM sleep) is necessary to bring about these representational changes. | 69/69 | Primary Analysis | Shared |
| Stat Learning | 10.15154/1504119 | Participants viewed centrally presented images, one at a time. There were 16 different images presented and each were composed of 8 image pairs. The pairs varied in visual similarity, from completely dissimilar to almost exactly the same. Participants were asked to identify when this square was present by pressing a key with their right index finger. Images were presented for 1 second each, and the ITI was 2, 4 or 6 s. Participants completed 8 runs of this task in the scanner, each of which was just over 5 minutes in length. in the first and last run, the images were presented in completely random order. In the remaining six runs, the images were presented as AB pairs, such that when the first member of a pair was presented, it was always followed by the second member of the pair. | 41/41 | Primary Analysis | Shared |
| Reductions in Retrieval Competition Predict the Benefit of Repeated Testing | 10.15154/1466212 | Repeated testing leads to improved long-term memory retention compared to repeated study, but the mechanism underlying this improvement remains controversial. In this work, we test the hypothesis that retrieval practice benefits subsequent recall by reducing competition from related memories. This hypothesis implies that the degree of reduction in competition between retrieval practice attempts should predict subsequent memory for practiced items. To test this prediction, we collected electroencephalography (EEG) data across two sessions. In the first session, participants practiced selectively retrieving exemplars from superordinate semantic categories (high competition), as well as retrieving the names of the superordinate categories from exemplars (low competition). In the second session, participants repeatedly studied and were tested on Swahili-English vocabulary. One week after session two, participants were again tested on the vocabulary. We trained a within-subject classifier on the data from session one to distinguish high and low competition states. We then used this classifier to measure the change in competition across multiple successful retrieval practice attempts in the second session. The degree to which competition decreased for a given vocabulary word predicted whether it was subsequently remembered in the third session. These results are consistent with the hypothesis that repeated testing improves retention by reducing competition. | 40/40 | Primary Analysis | Shared |
| Learning not to remember: How predicting the future impairs encoding of the present | 10.15154/1518488 | Memory is typically thought of as enabling reminiscence about past experiences. However, memory also informs and guides processing of future experiences. These two functions of memory are inherently incompatible: remembering specific experiences from the past requires storing idiosyncratic properties that define particular moments in space and time, but by definition such properties will not be shared with similar situations in the future and thus are not useful for prediction. We discovered that, when faced with this conflict, the brain prioritizes prediction over encoding. Behavioral tests of recognition and source recall showed that items allowing for prediction of what will appear next based on learned regularities were less likely to be encoded into memory. Brain imaging revealed that the hippocampus was responsible for this interference between statistical learning and episodic memory. The more that the hippocampus predicted the category of an upcoming item, the worse the current item was encoded. This competition may serve an adaptive purpose, focusing encoding on experiences for which we do not yet have a predictive model. | 36/36 | Primary Analysis | Shared |
| Neural overlap in item representations across episodes impairs context memory | 10.5072/1357077 | We frequently encounter the same item in different contexts, and when that happens, memories of earlier encounters can get reactivated in the brain. Here we examined how these existing memories are changed as a result of such reactivation. We hypothesized that when an item’s initial and subsequent neural representations overlap, this allows the initial item to become associated with novel contextual information, interfering with later retrieval of the initial context. That is, we predicted a negative relationship between representational similarity across repeated experiences of an item and subsequent source memory for the initial context. We tested this hypothesis in an fMRI study, in which objects were presented multiple times during different tasks. We measured the similarity of the neural patterns in lateral occipital cortex that were elicited by the first and second presentations of objects, and related this neural overlap score to source memory in a subsequent test. Consistent with our hypothesis, greater item-specific pattern similarity was linked to worse source memory for the initial task. Our findings suggest that the influence of novel experiences on an existing context memory depends on how reliably a shared component (i.e., same item) is represented across these episodes. | 32/32 | Primary Analysis | Shared |
| Violation Differentiation | 10.15154/1356633 | When an item is predicted in a particular context but the prediction is violated, memory for that item is weakened (Kim et al., 2014). Here, we explore what happens when such previously mispredicted items are later reencountered. According to prior neural network simulations, this sequence of events-misprediction and subsequent restudy-should lead to differentiation of the item's neural representation from the previous context (on which the misprediction was based). Specifically, misprediction weakens connections in the representation to features shared with the previous context and restudy allows new features to be incorporated into the representation that are not shared with the previous context. This cycle of misprediction and restudy should have the net effect of moving the item's neural representation away from the neural representation of the previous context. We tested this hypothesis using human fMRI by tracking changes in item-specific BOLD activity patterns in the hippocampus, a key structure for representing memories and generating predictions. In left CA2/3/DG, we found greater neural differentiation for items that were repeatedly mispredicted and restudied compared with items from a control condition that was identical except without misprediction. We also measured prediction strength in a trial-by-trial fashion and found that greater misprediction for an item led to more differentiation, further supporting our hypothesis. Therefore, the consequences of prediction error go beyond memory weakening. If the mispredicted item is restudied, the brain adaptively differentiates its memory representation to improve the accuracy of subsequent predictions and to shield it from further weakening. SIGNIFICANCE STATEMENT Competition between overlapping memories leads to weakening of nontarget memories over time, making it easier to access target memories. However, a nontarget memory in one context might become a target memory in another context. How do such memories get restrengthened without increasing competition again? Computational models suggest that the brain handles this by reducing neural connections to the previous context and adding connections to new features that were not part of the previous context. The result is neural differentiation away from the previous context. Here, we provide support for this theory, using fMRI to track neural representations of individual memories in the hippocampus and how they change based on learning.
| 32/32 | Primary Analysis | Shared |
| Relating Visual Production and Recognition of Objects in Human Visual Cortex | 10.15154/1503350 | Drawing is a powerful tool that can be used to convey rich perceptual information about objects in the world. What are the neural mechanisms that enable us to produce a recognizable drawing of an object, and how does this visual production experience influence how this object is represented in the brain? Here we evaluate the hypothesis that producing and recognizing an object recruit a shared neural representation, such that repeatedly drawing the object can enhance its perceptual discriminability in the brain. We scanned human participants (N = 31; 11 male) using fMRI across three phases of a training study: during training, participants repeatedly drew two objects in an alternating sequence on an MR-compatible tablet; before and after training, they viewed these and two other control objects, allowing us to measure the neural representation of each object in visual cortex. We found that: (1) stimulus-evoked representations of objects in visual cortex are recruited during visually cued production of drawings of these objects, even throughout the period when the object cue is no longer present; (2) the object currently being drawn is prioritized in visual cortex during drawing production, while other repeatedly drawn objects are suppressed; and (3) patterns of connectivity between regions in occipital and parietal cortex supported enhanced decoding of the currently drawn object across the training phase, suggesting a potential neural substrate for learning how to transform perceptual representations into representational actions. Together, our study provides novel insight into the functional relationship between visual production and recognition in the brain. SIGNIFICANCE STATEMENT: Humans can produce simple line drawings that capture rich information about their perceptual experiences. However, the mechanisms that support this behavior are not well understood. Here we investigate how regions in visual cortex participate in the recognition of an object and the production of a drawing of it. We find that these regions carry diagnostic information about an object in a similar format both during recognition and production, and that practice drawing an object enhances transmission of information about it to downstream regions. Together, our study provides novel insight into the functional relationship between visual production and recognition in the brain. | 31/31 | Primary Analysis | Shared |
| Associative prediction of visual shape in the hippocampus. | 10.15154/1469793 | The way we perceive the world is to a great extent determined by our prior knowledge. Despite this intimate link between perception and memory, these two aspects of cognition have mostly been studied in isolation. Here we investigate their interaction by asking how memory systems that encode and retrieve associations can inform perception. We find that upon hearing a familiar auditory cue, the hippocampus represents visual information that had previously co-occurred with the cue, even when this expectation differs from what is currently visible. Furthermore, the strength of this hippocampal expectation correlates with facilitation of perceptual processing in visual cortex. These findings help bridge the gap between memory and sensory systems in the human brain. | 24/24 | Primary Analysis | Shared |
| Content-based Dissociation of Hippocampal Involvement in Prediction | 10.15154/1517668 | Recent work suggests that a key function of the hippocampus is to predict the future. This is thought to depend on its ability to bind inputs over time and space and to retrieve upcoming or missing inputs based on partial cues. In line with this, previous research has revealed prediction-related signals in the hippocampus for complex visual objects, such as fractals and abstract shapes. Implicit in such accounts is that these computations in the hippocampus reflect domain-general processes that apply across different types and modalities of stimuli. An alternative is that the hippocampus plays a more domain-specific role in predictive processing, with the type of stimuli being predicted determining its involvement. To investigate this, we compared hippocampal responses to auditory cues predicting abstract shapes (Experiment 1) versus oriented gratings (Experiment 2). We measured brain activity in male and female human participants using high-resolution fMRI, in combination with inverted encoding models to reconstruct shape and orientation information. Our results revealed that expectations about shape and orientation evoked distinct representations in the hippocampus. For complex shapes, the hippocampus represented which shape was expected, potentially serving as a source of top–down predictions. In contrast, for simple gratings, the hippocampus represented only unexpected orientations, more reminiscent of a prediction error. We discuss several potential explanations for this content-based dissociation in hippocampal function, concluding that the computational role of the hippocampus in predictive processing may depend on the nature and complexity of stimuli. | 24/24 | Primary Analysis | Shared |
| Hippocampal-neocortical interactions sharpen over time for predictive actions. | 10.15154/1504543 | When an action is familiar, we are able to anticipate how it will change the state of the world. These expectations can result from retrieval of action-outcome associations in the hippocampus and the reinstatement of anticipated outcomes in visual cortex. How does this role for the hippocampus in action-based prediction change over time? We use high-resolution fMRI and a dual-training behavioral paradigm to examine how the hippocampus interacts with visual cortex during predictive and nonpredictive actions learned either three days earlier or immediately before the scan. Just-learned associations led to comparable background connectivity between the hippocampus and V1/V2, regardless of whether actions predicted outcomes. However, three-day-old associations led to stronger background connectivity and greater differentiation between neural patterns for predictive vs. nonpredictive actions. Hippocampal prediction may initially reflect indiscriminate binding of co-occurring events, with action information pruning weaker associations and leading to more selective and accurate predictions over time. | 24/24 | Primary Analysis | Shared |
| Human hippocampal replay during rest prioritizes weakly-learned information and predicts memory performance | 10.15154/1463093 | There is now extensive evidence that the hippocampus replays experiences during
quiet rest periods, and that this replay benefits subsequent memory. A critical open
question is how memories are prioritized for replay during these offline periods. We
addressed this question in an experiment in which participants learned the features of 15
objects and then underwent fMRI scanning to track item-level replay in the hippocampus
using pattern analysis during a rest period. Objects that were remembered less well were
replayed more during the subsequent rest period, suggesting a prioritization process in
which weaker memories—memories most vulnerable to forgetting—are selected for
wake replay. Participants came back for a second session, either after a night of sleep or a
day awake, and underwent another scanned rest period followed by a second memory
test. In the second session, more hippocampal replay of a satellite during the rest period
predicted better subsequent memory for that satellite. Only in the group with intervening
sleep did rest replay predict improvement from the first to second session. Our results
provide the first evidence that replay of individual memories occurs during rest in the
human hippocampus and that this replay prioritizes weakly learned information, predicts
subsequent memory performance, and relates to memory improvement across a delay
with sleep. | 24/24 | Primary Analysis | Shared |
| Sculpting New Visual Concepts into the Human Brain | 10.15154/1520878 | Learning requires changing the brain. This typically occurs through experience, study, or instruction. We report a new way of acquiring conceptual knowledge by directly sculpting activity patterns in the human brain. We used a non-invasive technique (closed-loop real-time functional magnetic resonance imaging) to create novel categories of visual objects in the brain. After training, participants exhibited behavioral and neural biases for the sculpted, but not control categories. The ability to sculpt new conceptual distinctions in the human brain, applied here to perception, has broad relevance to other domains of cognition such as decision-making, memory, and motor control. As such, the work opens up new frontiers in brain-machine interface design, neuroprosthetics, and neurorehabilitation. | 10/10 | Primary Analysis | Shared |
