The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall

The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall Figure

Abstract

The method of loci (MoL) is a well-known mnemonic technique in which visuospatial spatial environments are used to scaffold the memorization of non-spatial information. We developed a novel virtual reality-based implementation of the MoL in which participants used three unique virtual environments to serve as their “memory palaces.” In each world, participants were presented with a sequence of 15 3D objects that appeared in front of their avatar for 20 s each. The experimental group (N = 30) was given the ability to click on each object to lock it in place, whereas the control group (N = 30) was not afforded this functionality. We found that despite matched engagement, exposure duration, and instructions emphasizing the efficacy of the mnemonic across groups, participants in the experimental group recalled 28% more objects. We also observed a strong relationship between spatial memory for objects and landmarks in the environment and verbal recall strength. These results provide evidence for spatially mediated processes underlying the effectiveness of the MoL and contribute to theoretical models of memory that emphasize spatial encoding as the primary currency of mnemonic function.

Author’s Comments

This study was designed to both (a) test the hypothesis that the binding of information to a spatial scaffolding underlies the effectiveness of the MoL and (b) provide proof-of-concept for a user-friendly technology that mandates subject compliance in use of the MoL. The current investigation leverages virtual reality (VR), allowing participants to readily implement an MoL-based encoding strategy without the reliance on mental imagery. By providing a novel and common set of environments for participants, this study’s VR-based paradigm mitigates the discussed concerns regarding individual differences in mental imagery, environmental size, complexity, and exposure time. Furthermore, VR serves as a particularly viable medium for increasing the ecological validity of memory experiments in general (Reggente et al. 2018) and allows for the control and capture of experimental details (e.g., exposure time and place of each seen object).

The findings add important empirical evidence to the conversation surrounding the primacy of spatial contexts in encoding (see Robin 2018) and the recruitment of spatial processing codes for cognition (Bellmund et al. 2018).

Featured Result

Spatial memory for both objects encountered in the virtual environments (as determined by participant placement on a bird’s eye view of the map following encoding)correlated strongly with free recall memory for those same objects. Spatial memory for landmarks found within the environment also showed a relationship with free recall.

method of loci in virtual reality

Get The Article

Journal of Cognitive Enhancement

The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall

PDF

The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall

DOI

The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall

Citation

MLA

Reggente, Nicco, et al. “The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall.” Journal of Cognitive Enhancement (2019): 1-19.

APA

Reggente, N., Essoe, J. K., Baek, H. Y., & Rissman, J. (2019). The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall. Journal of Cognitive Enhancement, 1-19.

Chicago

Reggente, Nicco, Joey KY Essoe, Hera Younji Baek, and Jesse Rissman. “The Method of Loci in Virtual Reality: Explicit Binding of Objects to Spatial Contexts Enhances Subsequent Memory Recall.” Journal of Cognitive Enhancement (2019): 1-19.

Altmetric

https://springeropen.altmetric.com/details/63279608

Enhancing the Ecological Validity of fMRI Memory Research Using Virtual Reality

Authors: Nicco Reggente, Joey K.Y. Essoe, Zahra M. Aghajan, Amir V. Tavakoli, Joseph F. McGuire, Nanthia A. Suthana, Jesse Rissman

Article: PDF

Plain English: Functional magnetic resonance imaging (fMRI) is a powerful research tool to understand the neural underpinnings of human memory. However, as memory is known to be context-dependent, differences in contexts between naturalistic settings and the MRI scanner environment may potentially confound neuroimaging findings. Virtual reality (VR) provides a unique opportunity to mitigate this issue by allowing memories to be formed and/or retrieved within immersive, navigable, visuospatial contexts. This can enhance the ecological validity of task paradigms, while still ensuring that researchers maintain experimental control over critical aspects of the learning and testing experience. This mini-review surveys the growing body of fMRI studies that have incorporated VR to address critical questions about human memory. These studies have adopted a variety of approaches, including presenting research participants with VR experiences in the scanner, asking participants to retrieve information that they had previously acquired in a VR environment, or identifying neural correlates of behavioral metrics obtained through VR-based tasks performed outside the scanner. Although most such studies to date have focused on spatial or navigational memory, we also discuss the promise of VR in aiding other areas of memory research and facilitating research into clinical disorders.

Citation: Reggente, N., Essoe, J. K. Y., Aghajan, Z. M., Tavakoli, A. V., McGuire, J. F., Suthana, N. A., & Rissman, J. (2018). Enhancing the ecological validity of fMRI memory research using virtual reality. Frontiers in Neuroscience, 12, 408.

Frontiers in Neuroscience Hosting: https://www.frontiersin.org/articles/10.3389/fnins.2018.00408/full

Altmetric: https://frontiers.altmetric.com/details/42679981

Memory Recall for High Reward Value Items Correlates With Individual Differences in White Matter Pathways Associated With Reward Processing and Fronto-Temporal Communication

Authors: Nicco Reggente, Michael S. Cohen, (Amy) Zhong S. Zheng, Alan D. Castel, Barbara J. Knowlton, Jesse Rissman

Article: PDF

Plain English: We observed individual differences in a value-directed remembering task. White matter integrity within the Uncinate Fasciculus correlated with memory strength for high-value items. This relationship was also seen for the tract connecting the Nucleus Accumbens (NAcc) and Ventral Tegmental Area (VTA). Additionally, structural integrity of the NAcc –> VTA tract correlated with the degree an individual was selective about the information they retained (i.e. preferential recall for high-value over low-value items).

Citation: Reggente, N., Cohen, M. S., Castel, A., Knowlton, B. J., & Rissman, J. (2018). Memory recall for high reward value items correlates with individual differences in white matter pathways associated with reward processing and fronto-temporal communication. Frontiers in Human Neuroscience, 12, 241.

Frontiers in Human Neuroscience Hosting: https://www.frontiersin.org/articles/10.3389/fnhum.2018.00241/full

Altmetric: https://frontiers.altmetric.com/details/42694375

Multivariate resting-state functional connectivity predicts response to cognitive behavioral therapy in obsessive – compulsive disorder

Authors: Nicco Reggente, Teena D. Moody, Francesca Morfini, Courtney Sheen, Jesse Rissman, Joseph O’Neill and Jamie D. Feusner
Plain English: In this work, we collected Resting-State fMRI data from patients with OCD before they engaged in 4 weeks of Cognitive Behavioral Therapy (CBT). We calculated functional connectivity within the Default Mode and Visual Networks and trained a machine-learning classifier to learn how those patterns relate to a patient’s success in the CBT. The classifier was able to learn so well from these patterns that it could predict a patient’s OCD symptom severity (as measured by YBOCS) after CBT– effectively predicting their symptoms 4 weeks in the future. Such insights could help guide treatment options.
Citation: Reggente, N., Moody, T. D., Morfini, F., Sheen, C., Rissman, J., O’Neill, J., & Feusner, J. D. (2018). Multivariate resting-state functional connectivity predicts response to cognitive behavioral therapy in obsessive–compulsive disorder. Proceedings of the National Academy of Sciences, 201716686. https://doi.org/10.1073/pnas.1716686115

The Method of Loci revisited: Virtually augmented memory palaces [ICPS 2017 Symposium Presentation]

Symposium talk presented at ICPS 2017 in Vienna, Austria

Abstract:

Humans have long appreciated that visuospatial cues can serve as a scaffolding for the encoding of non-spatial content. The Method of Loci (MoL), which binds objects to a spatial context in one’s mental imagery, has helped enhance the mnemonic retrieval processes of memory champions since Ancient Greece. An adaptation of this method that seamlessly extends such benefits to the mass market could revolutionize how we process and convey information. At the forefront of such development is a startup company named Altar that has has begun the productization of a virtual Method of Loci, called Altar Show. This novel virtual reality presentation software is already being used in contexts ranging from business pitches, to secondary education and employee training.

Presentation: Reggente_ICPS_2017_Symposium

Prediction of response to cognitive-behavioral therapy in obsessive-compulsive disorder: a multivariate analysis of resting state functional connectivity

Jamie D Feusner, MD1; Nicco Reggente, MA2; Teena D Moody, PhD1; Francesca Morfini, MA1; Jesse Rissman, PhD1,2; Joseph O’Neill, PhD1

Affiliation:

1Department of Psychiatry and Biobehavioral Sciences, David Geffen School of Medicine at UCLA, Los Angeles, California 2Department of Psychology, UCLA, Los Angeles, California

Background: Cognitive-behavioral therapy (CBT) is an effective treatment for reducing symptoms of obsessive-compulsive disorder (OCD). Although many with OCD will benefit from CBT, the response still varies significantly between individuals. In addition, specialized CBT for OCD has limited availability, can be an expensive treatment, and by its nature is stressful and often time-consuming. This underscores the importance of developing reliable predictors of response to treatment to help with clinical decision-making. Although several studies have examined clinical and neurobiological features pre-treatment that are correlated with response to treatment, only one has examined functional connectivity as a predictor, and none have applied multivariate approaches. We used a multivariate pattern recognition approach applied to resting state functional connectivity pre-CBT in order to make predictive inferences on the individual patient level, as to their degree of response to treatment. In addition, we applied the same approaches to pre-treatment symptomatology in order to further elucidate mechanisms of functional connectivity associated with obsessions and compulsions, in a data-driven manner.

Methods: We acquired resting state functional magnetic resonance image BOLD data in 25 medicated and unmedicated adults with OCD before 4 weeks of intensive daily exposure and response prevention, a form of CBT. Core OCD symptomatology was measured using the Yale-Brown Obsessive Compulsive Scale (YBOCS). Image preprocessing included parcellation of the brain into 264 regions of interest, each belonging to one of 14 functional networks previously derived from meta-analyses of functional studies. We computed a pairwise Pearson-correlation matrix for each mean time course resulting in a 264 x 264 matrix containing the pairwise functional connectivity values (r-values) across all ROIs. Matrix cells corresponding to each functional network were identified to create feature sets. We implemented a leave-one-patient-out cross-validation to assess the predictive power of our feature sets in regards to our behavioral measures of interest: change in YBOCS scores from pre- to post-CBT. Specifically, we built a least absolute shrinkage and selection operator (LASSO) regression model on n-1 patients using their feature sets. We correlated the predicted values with the actual values in order to yield a multiple R2 as a measure of our model’s feature-dependent predictivity. Additionally, we applied the same analysis to the pre-CBT (baseline) YBOCS scores.

Results: OCD participants showed significant clinical symptom improvements pre- to post-CBT (YBOCS scores X±Y pre-CBT; Z±Q post-CBT; t26=P, p<.R). Connectivity strength in the ventral attention network predicted greater/lesser reduction of YBOCS scores pre- to post-CBT ( =.185, P=.01. Connectivity strength in the cingulo-opercular network at baseline was predictive of baseline severity of YBOCS scores ( =.35, P=.0009).

Conclusions: This represents the first study in OCD to use multivariate pattern recognition approaches to determine neurobiological markers predictive of response to treatment. Strength of resting state functional connectivity in the ventral attention network was associated with a better response to treatment. This may signify that those with better inherent ability to attend to perceptually-driven stimuli in their environment (perhaps also reflecting that they are less internally distracted by obsessive thoughts) may respond better to treatment. In addition, the phenomenology of obsessions and compulsions, specifically before treatment, is associated with connectivity in the cingulo-opercular network. Given the function of this network, those with weaker connectivity may be less able to maintain control over behaviors and thought patterns in the face of emotional arousal, and hence have higher degree of obsessions and compulsions. Results have clinical implications for identifying individual OCD patients who will maximally benefit from treatment with intensive CBT, and have implications for further understanding the pathophysiology of OCD.

View the Poster, Presented at ACNP (2016)

The Method of Loci revisited: Memory enhancement by way of virtually augmented memory palaces

Reggente, N., Essoe, J., Mehta, P.*, Ohno, A.*, Rissman, J.

Humans have long appreciated that visuospatial cues can serve as a scaffolding for the encoding of non-spatial content. The Method of Loci (MoL), which binds objects to a spatial context in one’s mental imagery, has been the favored mnemonic strategy of memory champions since Ancient Greece. In this work, we created a virtual reality implementation of the MoL to tease apart the factors that contribute to the MoL’s undeniable efficacy as a memory enhancement technique. We crafted three distinct virtual environments where subjects could view objects. Subjects that were told to place items at locations of their choosing recalled significantly more objects than subjects who only viewed the objects. We also addressed the contributions of volition and contextual richness to recall strength.

View the poster, presented at ICOM in Budapest (2016)

Neural correlates of fluid intelligence via functional and structural network connectivity measures

Connectivity across regions in the brain can be characterized as either functional (correlated fluctuations in activity as measured by resting-state fMRI data) or structural (white matter pathways as measured by diffusion MRI data). Emerging studies suggest that the connections across brain regions that make up distinct cognitive networks can partially explain individual differences in behavioral traits. Some theorize that a reliable benchmark of intelligence is the ability to identify subtle patterns across distantly related ideas. The Raven’s Progressive Matrices (RPM), a pattern completion task, is one widely used measure of general fluid intelligence. Here, we use a combination of functional and structural connectivity metrics derived from a large MRI dataset [n=127] to examine the relationship between neural connectivity and RPM scores. We used a Support Vector Regression cross-validation procedure to assess the degree to which we could predict a subject’s intelligence based on these connectivity values. We were able to account for 14% of the variance in individuals’ intelligence scores when using specific combinations of functional and structural connectivity values.

You can view our poster here:

Vuong, Reggente, Rissman Poster Presented at UCLA PURC 2016

Disentangling Disorders of Consciousness: Insights from Diffusion Tensor Imaging and Machine Learning

Abstract: Previous studies have suggested that disorders of consciousness (DOC) after severe brain injury may result from disconnections of the thalamo-cortical system. However, thalamo-cortical connectivity differences between vegetative state (VS), minimally conscious state minus (MCS-,i.e., low-level behavior such as visual pursuit), and minimally conscious state plus (MCS+, i.e., high-level behavior such as language processing) remain unclear. We employed probabilistic tractography in a sample of 25 DOC patients to assess whether structural connectivity in various thalamo-cortical circuits could differentiate between VS, MCS-, and MCS+ patients. First, we individually segmented the thalamus into seven clusters based on patterns of cortical connectivity and tested for univariate differences across groups. Second, reconstructed wholebrain thalamic tracks were used as features in a multivariate searchlight analysis to identify regions along the tracks that were most informative in distinguishing among groups. At the univariate level, we found that VS patients displayed reduced connectivity in most thalamocortical circuits of interest, including frontal, temporal, and sensorimotor connections, as compared to MCS+, but showed more pulvinar-occipital connections when compared to MCS-.Moreover, MCS- exhibited significantly less thalamo-premotor and thalamo-temporal connectivity than MCS+. At the multivariate level, we found that thalamic tracks reaching frontal, parietal, and sensorimotor regions, could discriminate, up to 100% accuracy, across each pairwise group comparison. Together, these findings highlight the role of thalamo-cortical connections in patients’ behavioral profile and level of consciousness. Diffusion tensor imaging combined with machine learning algorithms could thus potentially facilitate diagnostic distinctions in DOC and shed light on the neural correlates of consciousness.

Disentangling Disorders of Consciousness: Insights from Diffusion Tensor Imaging and Machine Learning — Human Brain Mapping (2016)

Mapping neural representations of heading direction and environmental context during imagined navigation of learned virtual environments

Authors: Nicco Reggente., Essoe, J.K., Jevtic, I., Rissman, J.

Abstract for CNS 2016 in New York, NY: Constructing a rich egocentric representation of one’s movement about an environment is a multi-faceted effort requiring a vast interplay across cortical areas responsible for visual processing, heading direction, and spatial coding. While electrophysiological recordings in rodents have identified robust neural correlates related to distinct aspects of navigation, experimental work with human subjects offers the unique potential to elucidate the mechanisms of navigational mental imagery, a process we frequently engage in when planning a route or giving directions. In the present study, we first familiarized participants with navigational paths about three highly distinctive virtual environments. The next day, while undergoing fMRI scanning, participants viewed a series of first-person videos that indicated either clockwise or counter-clockwise movement around the perimeter of each environment. After several rounds of video viewing, participants performed a new task in which they were covertly cued to imagine themselves walking along each of these same routes. We leveraged support vector machines within a searchlight-mapping approach to identify brain regions whose BOLD patterns coded for information pertaining to the participants’ heading direction or environmental context. As anticipated, many visual association regions with significant accuracy for decoding the contents of perceived navigation were also capable of decoding imagined navigation, although imagery classification performance was generally less robust. Interestingly, several frontal and temporal lobe regions showed decoding effects that were specific to mental imagery, and the distribution of these areas differed as a function of gender, potentially indicative of a qualitatively different mental representation of navigational information across males and females.

The full poster can be seen here: Reggente_et_al_CNS_Final