The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall
Posted on July 21, 2019
The method of loci (MoL) is a well-known mnemonic technique in which visuospatial spatial environments are used to scaffold the memorization of non-spatial information. We developed a novel virtual reality-based implementation of the MoL in which participants used three unique virtual environments to serve as their “memory palaces.” In each world, participants were presented with a sequence of 15 3D objects that appeared in front of their avatar for 20 s each. The experimental group (N = 30) was given the ability to click on each object to lock it in place, whereas the control group (N = 30) was not afforded this functionality. We found that despite matched engagement, exposure duration, and instructions emphasizing the efficacy of the mnemonic across groups, participants in the experimental group recalled 28% more objects. We also observed a strong relationship between spatial memory for objects and landmarks in the environment and verbal recall strength. These results provide evidence for spatially mediated processes underlying the effectiveness of the MoL and contribute to theoretical models of memory that emphasize spatial encoding as the primary currency of mnemonic function.
This study was designed to both (a) test the hypothesis that the binding of information to a spatial scaffolding underlies the effectiveness of the MoL and (b) provide proof-of-concept for a user-friendly technology that mandates subject compliance in use of the MoL. The current investigation leverages virtual reality (VR), allowing participants to readily implement an MoL-based encoding strategy without the reliance on mental imagery. By providing a novel and common set of environments for participants, this study’s VR-based paradigm mitigates the discussed concerns regarding individual differences in mental imagery, environmental size, complexity, and exposure time. Furthermore, VR serves as a particularly viable medium for increasing the ecological validity of memory experiments in general (Reggente et al. 2018) and allows for the control and capture of experimental details (e.g., exposure time and place of each seen object).
The findings add important empirical evidence to the conversation surrounding the primacy of spatial contexts in encoding (see Robin 2018) and the recruitment of spatial processing codes for cognition (Bellmund et al. 2018).
Spatial memory for both objects encountered in the virtual environments (as determined by participant placement on a bird’s eye view of the map following encoding)correlated strongly with free recall memory for those same objects. Spatial memory for landmarks found within the environment also showed a relationship with free recall.
Get The Article
Journal of Cognitive Enhancement
Reggente, Nicco, et al. “The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall.” Journal of Cognitive Enhancement (2019): 1-19.
Reggente, N., Essoe, J. K., Baek, H. Y., & Rissman, J. (2019). The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall. Journal of Cognitive Enhancement, 1-19.
Reggente, Nicco, Joey KY Essoe, Hera Younji Baek, and Jesse Rissman. “The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall.” Journal of Cognitive Enhancement (2019): 1-19.
Posted on June 15, 2018
Authors: Nicco Reggente, Joey K.Y. Essoe, Zahra M. Aghajan, Amir V. Tavakoli, Joseph F. McGuire, Nanthia A. Suthana, Jesse Rissman
Plain English: Functional magnetic resonance imaging (fMRI) is a powerful research tool to understand the neural underpinnings of human memory. However, as memory is known to be context-dependent, differences in contexts between naturalistic settings and the MRI scanner environment may potentially confound neuroimaging findings. Virtual reality (VR) provides a unique opportunity to mitigate this issue by allowing memories to be formed and/or retrieved within immersive, navigable, visuospatial contexts. This can enhance the ecological validity of task paradigms, while still ensuring that researchers maintain experimental control over critical aspects of the learning and testing experience. This mini-review surveys the growing body of fMRI studies that have incorporated VR to address critical questions about human memory. These studies have adopted a variety of approaches, including presenting research participants with VR experiences in the scanner, asking participants to retrieve information that they had previously acquired in a VR environment, or identifying neural correlates of behavioral metrics obtained through VR-based tasks performed outside the scanner. Although most such studies to date have focused on spatial or navigational memory, we also discuss the promise of VR in aiding other areas of memory research and facilitating research into clinical disorders.
Citation: Reggente, N., Essoe, J. K. Y., Aghajan, Z. M., Tavakoli, A. V., McGuire, J. F., Suthana, N. A., & Rissman, J. (2018). Enhancing the ecological validity of fMRI memory research using virtual reality. Frontiers in Neuroscience, 12, 408.
Frontiers in Neuroscience Hosting: https://www.frontiersin.org/articles/10.3389/fnins.2018.00408/full
Memory Recall for High Reward Value Items Correlates With Individual Differences in White Matter Pathways Associated With Reward Processing and Fronto-Temporal Communication
Posted on June 15, 2018
Authors: Nicco Reggente, Michael S. Cohen, (Amy) Zhong S. Zheng, Alan D. Castel, Barbara J. Knowlton, Jesse Rissman
Plain English: We observed individual differences in a value-directed remembering task. White matter integrity within the Uncinate Fasciculus correlated with memory strength for high-value items. This relationship was also seen for the tract connecting the Nucleus Accumbens (NAcc) and Ventral Tegmental Area (VTA). Additionally, structural integrity of the NAcc –> VTA tract correlated with the degree an individual was selective about the information they retained (i.e. preferential recall for high-value over low-value items).
Citation: Reggente, N., Cohen, M. S., Castel, A., Knowlton, B. J., & Rissman, J. (2018). Memory recall for high reward value items correlates with individual differences in white matter pathways associated with reward processing and fronto-temporal communication. Frontiers in Human Neuroscience, 12, 241.
Frontiers in Human Neuroscience Hosting: https://www.frontiersin.org/articles/10.3389/fnhum.2018.00241/full
Multivariate resting-state functional connectivity predicts response to cognitive behavioral therapy in obsessive – compulsive disorder
Posted on February 13, 2018
Symposium talk presented at ICPS 2017 in Vienna, Austria
Humans have long appreciated that visuospatial cues can serve as a scaffolding for the encoding of non-spatial content. The Method of Loci (MoL), which binds objects to a spatial context in one’s mental imagery, has helped enhance the mnemonic retrieval processes of memory champions since Ancient Greece. An adaptation of this method that seamlessly extends such benefits to the mass market could revolutionize how we process and convey information. At the forefront of such development is a startup company named Altar that has has begun the productization of a virtual Method of Loci, called Altar Show. This novel virtual reality presentation software is already being used in contexts ranging from business pitches, to secondary education and employee training.
Prediction of response to cognitive-behavioral therapy in obsessive-compulsive disorder: a multivariate analysis of resting state functional connectivity
Jamie D Feusner, MD1; Nicco Reggente, MA2; Teena D Moody, PhD1; Francesca Morfini, MA1; Jesse Rissman, PhD1,2; Joseph O’Neill, PhD1
1Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine at UCLA, Los Angeles, California 2Department of Psychology, UCLA, Los Angeles, California
Background: Cognitive-behavioral therapy (CBT) is an effective treatment for reducing symptoms of obsessive-compulsive disorder (OCD). Although many with OCD will benefit from CBT, the response still varies significantly between individuals. In addition, specialized CBT for OCD has limited availability, can be an expensive treatment, and by its nature is stressful and often time-consuming. This underscores the importance of developing reliable predictors of response to treatment to help with clinical decision-making. Although several studies have examined clinical and neurobiological features pre-treatment that are correlated with response to treatment, only one has examined functional connectivity as a predictor, and none have applied multivariate approaches. We used a multivariate pattern recognition approach applied to resting state functional connectivity pre-CBT in order to make predictive inferences on the individual patient level, as to their degree of response to treatment. In addition, we applied the same approaches to pre-treatment symptomatology in order to further elucidate mechanisms of functional connectivity associated with obsessions and compulsions, in a data-driven manner.
Methods: We acquired resting state functional magnetic resonance image BOLD data in 25 medicated and unmedicated adults with OCD before 4 weeks of intensive daily exposure and response prevention, a form of CBT. Core OCD symptomatology was measured using the Yale-Brown Obsessive Compulsive Scale (YBOCS). Image preprocessing included parcellation of the brain into 264 regions of interest, each belonging to one of 14 functional networks previously derived from meta-analyses of functional studies. We computed a pairwise Pearson-correlation matrix for each mean time course resulting in a 264 x 264 matrix containing the pairwise functional connectivity values (r-values) across all ROIs. Matrix cells corresponding to each functional network were identified to create feature sets. We implemented a leave-one-patient-out cross-validation to assess the predictive power of our feature sets in regards to our behavioral measures of interest: change in YBOCS scores from pre- to post-CBT. Specifically, we built a least absolute shrinkage and selection operator (LASSO) regression model on n-1 patients using their feature sets. We correlated the predicted values with the actual values in order to yield a multiple R2 as a measure of our model’s feature-dependent predictivity. Additionally, we applied the same analysis to the pre-CBT (baseline) YBOCS scores.
Results: OCD participants showed significant clinical symptom improvements pre- to post-CBT (YBOCS scores X±Y pre-CBT; Z±Q post-CBT; t26=P, p<.R). Connectivity strength in the ventral attention network predicted greater/lesser reduction of YBOCS scores pre- to post-CBT ( =.185, P=.01. Connectivity strength in the cingulo-opercular network at baseline was predictive of baseline severity of YBOCS scores ( =.35, P=.0009).
Conclusions: This represents the first study in OCD to use multivariate pattern recognition approaches to determine neurobiological markers predictive of response to treatment. Strength of resting state functional connectivity in the ventral attention network was associated with a better response to treatment. This may signify that those with better inherent ability to attend to perceptually-driven stimuli in their environment (perhaps also reflecting that they are less internally distracted by obsessive thoughts) may respond better to treatment. In addition, the phenomenology of obsessions and compulsions, specifically before treatment, is associated with connectivity in the cingulo-opercular network. Given the function of this network, those with weaker connectivity may be less able to maintain control over behaviors and thought patterns in the face of emotional arousal, and hence have higher degree of obsessions and compulsions. Results have clinical implications for identifying individual OCD patients who will maximally benefit from treatment with intensive CBT, and have implications for further understanding the pathophysiology of OCD.
Reggente, N., Essoe, J., Mehta, P.*, Ohno, A.*, Rissman, J.
Humans have long appreciated that visuospatial cues can serve as a scaffolding for the encoding of non-spatial content. The Method of Loci (MoL), which binds objects to a spatial context in one’s mental imagery, has been the favored mnemonic strategy of memory champions since Ancient Greece. In this work, we created a virtual reality implementation of the MoL to tease apart the factors that contribute to the MoL’s undeniable efficacy as a memory enhancement technique. We crafted three distinct virtual environments where subjects could view objects. Subjects that were told to place items at locations of their choosing recalled significantly more objects than subjects who only viewed the objects. We also addressed the contributions of volition and contextual richness to recall strength.
Connectivity across regions in the brain can be characterized as either functional (correlated fluctuations in activity as measured by resting-state fMRI data) or structural (white matter pathways as measured by diffusion MRI data). Emerging studies suggest that the connections across brain regions that make up distinct cognitive networks can partially explain individual differences in behavioral traits. Some theorize that a reliable benchmark of intelligence is the ability to identify subtle patterns across distantly related ideas. The Raven’s Progressive Matrices (RPM), a pattern completion task, is one widely used measure of general fluid intelligence. Here, we use a combination of functional and structural connectivity metrics derived from a large MRI dataset [n=127] to examine the relationship between neural connectivity and RPM scores. We used a Support Vector Regression cross-validation procedure to assess the degree to which we could predict a subject’s intelligence based on these connectivity values. We were able to account for 14% of the variance in individuals’ intelligence scores when using specific combinations of functional and structural connectivity values.
You can view our poster here:
Disentangling Disorders of Consciousness: Insights from Diffusion Tensor Imaging and Machine Learning
Abstract: Previous studies have suggested that disorders of consciousness (DOC) after severe brain injury may result from disconnections of the thalamo-cortical system. However, thalamo-cortical connectivity differences between vegetative state (VS), minimally conscious state minus (MCS-,i.e., low-level behavior such as visual pursuit), and minimally conscious state plus (MCS+, i.e., high-level behavior such as language processing) remain unclear. We employed probabilistic tractography in a sample of 25 DOC patients to assess whether structural connectivity in various thalamo-cortical circuits could differentiate between VS, MCS-, and MCS+ patients. First, we individually segmented the thalamus into seven clusters based on patterns of cortical connectivity and tested for univariate differences across groups. Second, reconstructed wholebrain thalamic tracks were used as features in a multivariate searchlight analysis to identify regions along the tracks that were most informative in distinguishing among groups. At the univariate level, we found that VS patients displayed reduced connectivity in most thalamocortical circuits of interest, including frontal, temporal, and sensorimotor connections, as compared to MCS+, but showed more pulvinar-occipital connections when compared to MCS-.Moreover, MCS- exhibited significantly less thalamo-premotor and thalamo-temporal connectivity than MCS+. At the multivariate level, we found that thalamic tracks reaching frontal, parietal, and sensorimotor regions, could discriminate, up to 100% accuracy, across each pairwise group comparison. Together, these findings highlight the role of thalamo-cortical connections in patients’ behavioral profile and level of consciousness. Diffusion tensor imaging combined with machine learning algorithms could thus potentially facilitate diagnostic distinctions in DOC and shed light on the neural correlates of consciousness.
Mapping neural representations of heading direction and environmental context during imagined navigation of learned virtual environments
Authors: Nicco Reggente., Essoe, J.K., Jevtic, I., Rissman, J.
Abstract for CNS 2016 in New York, NY: Constructing a rich egocentric representation of one’s movement about an environment is a multi-faceted effort requiring a vast interplay across cortical areas responsible for visual processing, heading direction, and spatial coding. While electrophysiological recordings in rodents have identified robust neural correlates related to distinct aspects of navigation, experimental work with human subjects offers the unique potential to elucidate the mechanisms of navigational mental imagery, a process we frequently engage in when planning a route or giving directions. In the present study, we first familiarized participants with navigational paths about three highly distinctive virtual environments. The next day, while undergoing fMRI scanning, participants viewed a series of first-person videos that indicated either clockwise or counter-clockwise movement around the perimeter of each environment. After several rounds of video viewing, participants performed a new task in which they were covertly cued to imagine themselves walking along each of these same routes. We leveraged support vector machines within a searchlight-mapping approach to identify brain regions whose BOLD patterns coded for information pertaining to the participants’ heading direction or environmental context. As anticipated, many visual association regions with significant accuracy for decoding the contents of perceived navigation were also capable of decoding imagined navigation, although imagery classification performance was generally less robust. Interestingly, several frontal and temporal lobe regions showed decoding effects that were specific to mental imagery, and the distribution of these areas differed as a function of gender, potentially indicative of a qualitatively different mental representation of navigational information across males and females.
The full poster can be seen here: Reggente_et_al_CNS_Final