- Article
- Published:
- Diego Aldarondo ORCID: orcid.org/0000-0001-8558-75571,2nAff4,
- Josh Merel2nAff4,
- Jesse D. Marshall ORCID: orcid.org/0000-0003-4810-67121nAff5,
- Leonard Hasenclever2,
- Ugne Klibaite1,
- Amanda Gellis1,
- Yuval Tassa ORCID: orcid.org/0000-0002-1197-288X2,
- Greg Wayne2,
- Matthew Botvinick ORCID: orcid.org/0000-0001-7758-68962,3 &
- …
- Bence P. Ölveczky ORCID: orcid.org/0000-0003-2499-27051
Nature (2024)Cite this article
-
11k Accesses
-
609 Altmetric
-
Metrics details
We are providing an unedited version of this manuscript to give early access to its findings. Before final publication, the manuscript will undergo further editing. Please note there may be errors present which affect the content, and all legal disclaimers apply.
Subjects
- Biophysical models
- Motor control
- Network models
Abstract
Animals have exquisite control of their bodies, allowing them to perform a diverse range of behaviors. How such control is implemented by the brain, however, remains unclear. Advancing our understanding requires models that can relate principles of control to the structure of neural activity in behaving animals. To facilitate this, we built a ‘virtual rodent’, in which an artificial neural network actuates a biomechanically realistic model of the rat 1 in a physics simulator 2. We used deep reinforcement learning 3–5 to train the virtual agent to imitate the behavior of freely-moving rats, thus allowing us to compare neural activity recorded in real rats to the network activity of a virtual rodent mimicking their behavior. We found that neural activity in the sensorimotor striatum and motor cortex was better predicted by the virtual rodent’s network activity than by any features of the real rat’s movements, consistent with both regions implementing inverse dynamics 6. Furthermore, the network’s latent variability predicted the structure of neural variability across behaviors and afforded robustness in a way consistent with the minimal intervention principle of optimal feedback control 7. These results demonstrate how physical simulation of biomechanically realistic virtual animals can help interpret the structure of neural activity across behavior and relate it to theoretical principles of motor control.
This is a preview of subscription content, access via your institution
Access options
Change institution
Buy or subscribe
Access Nature and 54 other Nature Portfolio journals
Get Nature+, our best-value online-access subscription
$29.99 /30days
cancel any time
Learn more
Subscribe to this journal
Receive 51 print issues and online access
$199.00 per year
only $3.90 per issue
Learn more
Rent or buy this article
Prices vary by article type
from$1.95
to$39.95
Learn more
Prices may be subject to local taxes which are calculated during checkout
Similar content being viewed by others
Recurrent neural networks with explicit representation of dynamic latent variables can mimic behavioral patterns in a physical inference task
Article Open access 04 October 2022
Mesolimbic dopamine adapts the rate of learning from action
Article Open access 18 January 2023
Quantifying behavior to understand the brain
Article 09 November 2020
Author information
Author notes
Diego Aldarondo&Josh Merel
Present address: Fauna Robotics, 900 Broadway, Suite 903, New York, NY, USA
Jesse D. Marshall
Present address: Reality Labs, Meta, 421 8th Ave, New York, NY, USA
Authors and Affiliations
Department of Organismic and Evolutionary Biology and Center for Brain Science, Harvard University, Cambridge, MA, USA
Diego Aldarondo,Jesse D. Marshall,Ugne Klibaite,Amanda Gellis&Bence P. Ölveczky
DeepMind, Google, 5 New Street Square, London, UK
Diego Aldarondo,Josh Merel,Leonard Hasenclever,Yuval Tassa,Greg Wayne&Matthew Botvinick
Gatsby Computational Neuroscience Unit, University College London, 46 Cleveland Street, London, UK
Matthew Botvinick
Authors
- Diego Aldarondo
View author publications
See AlsoNeural DSP Quad Cortex reviewYou can also search for this author in PubMedGoogle Scholar
- Josh Merel
View author publications
You can also search for this author in PubMedGoogle Scholar
- Jesse D. Marshall
View author publications
You can also search for this author in PubMedGoogle Scholar
- Leonard Hasenclever
View author publications
You can also search for this author in PubMedGoogle Scholar
- Ugne Klibaite
View author publications
You can also search for this author in PubMedGoogle Scholar
- Amanda Gellis
View author publications
You can also search for this author in PubMedGoogle Scholar
- Yuval Tassa
View author publications
You can also search for this author in PubMedGoogle Scholar
- Greg Wayne
View author publications
You can also search for this author in PubMedGoogle Scholar
- Matthew Botvinick
View author publications
You can also search for this author in PubMedGoogle Scholar
- Bence P. Ölveczky
View author publications
You can also search for this author in PubMedGoogle Scholar
Corresponding authors
Correspondence to Diego Aldarondo or Bence P. Ölveczky.
Supplementary information
Supplementary Information
This file contains Supplementary Discussion and Supplementary Tables 1-3.
Supplementary Video 1
Overview of the MIMIC pipeline. The MIMIC pipeline consists of multi-camera video acquisition.
Supplementary Video 2
Accurate 3D pose estimation with DANNCE. We used DANNCE to estimate the 3D pose of freely moving rats from multi-camera recordings. This video depicts the DANNCE keypoint estimates overlain atop the original video recordings from all six cameras. Keypoint estimates are accurate across a wide range of behaviors.
Supplementary Video 3
Accurate skeletal registration with STAC. We used a custom implementation of simultaneous tracking and calibration (STAC).
Rights and permissions
About this article
Cite this article
Aldarondo, D., Merel, J., Marshall, J.D. et al. A virtual rodent predicts the structure of neural activity across behaviors. Nature (2024). https://doi.org/10.1038/s41586-024-07633-4
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41586-024-07633-4
Comments
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
