Biophysically Plausible Learning in the Brain via Eligibility Traces: Cortical Sequences, Hippocampal Place Cells, and Dopaminergic Reward Prediction Error

dc.contributor.advisorShouval, Harelen_US
dc.contributor.advisorPitkow, Xaqen_US
dc.creatorCone, Ianen_US
dc.date.accessioned2021-08-27T18:44:34Zen_US
dc.date.available2022-08-01T05:01:11Zen_US
dc.date.created2021-08en_US
dc.date.issued2021-08-25en_US
dc.date.submittedAugust 2021en_US
dc.date.updated2021-08-27T18:44:34Zen_US
dc.description.abstractThe brain’s ability to learn and associate temporally distal stimuli is one of its most fundamental (and puzzling) functions. The behaviorally relevant time scales (hundreds of milliseconds to seconds) at which the brain must link actions to reward are largely incompatible with Hebbian or STDP-like correlative learning rules. To solve this conundrum, we posit that two types of Hebbian activated, synapse specific “eligibility traces” – one associated with long term potentiation and the other long term depression – act as long lasting synaptic “tags” of previous activity . Upon presentation of a reinforcement signal, these two traces act in competition to determine long term changes in synaptic strength. In this work, we demonstrate the efficacy of this two-trace learning rule in three separate models. The first focuses on the learning and recall of uncompressed temporal sequences, based on recent experimental data from the visual cortex. The second model replicates so called “behavioral time scale plasticity” in hippocampal CA1, where the induction of a dendritic calcium spike triggers plasticity in place fields well in the past or future along the track traversal. Finally, this thesis showcases a model of dopaminergic cells demonstrating reward prediction error, including in the context of various “blocking” and “unblocking” paradigms. These models adhere to biophysical realism as much as possible; leaky-integrate-and-fire neurons with realistic noise are used when appropriate, and the models are either based on or replicate experimental results. Notably, and in contrast to many contemporary models which deal with the temporal credit assignment problem, eligibility traces allow for the principles of locality and causality to always be conserved. The success of these models presents a compelling case for the widespread utility of eligibility traces across a wide range of temporal tasks, and the models’ adherence to biophysical realism lend plausibility to the idea that eligibility traces are actually implemented in such a manner in the brain.en_US
dc.embargo.terms2022-08-01en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationCone, Ian. "Biophysically Plausible Learning in the Brain via Eligibility Traces: Cortical Sequences, Hippocampal Place Cells, and Dopaminergic Reward Prediction Error." (2021) Diss., Rice University. <a href="https://hdl.handle.net/1911/111339">https://hdl.handle.net/1911/111339</a>.en_US
dc.identifier.urihttps://hdl.handle.net/1911/111339en_US
dc.language.isoengen_US
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.subjectEligibility tracesen_US
dc.subjectdopamineen_US
dc.subjectVTAen_US
dc.subjecthippocampusen_US
dc.subjectsequence learningen_US
dc.subjectneural circuitsen_US
dc.subjectnon-Markovian sequencesen_US
dc.titleBiophysically Plausible Learning in the Brain via Eligibility Traces: Cortical Sequences, Hippocampal Place Cells, and Dopaminergic Reward Prediction Erroren_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentApplied Physicsen_US
thesis.degree.disciplineNatural Sciencesen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelDoctoralen_US
thesis.degree.majorApplied Physics/Electrical Engen_US
thesis.degree.nameDoctor of Philosophyen_US
Files
Original bundle
Now showing 1 - 1 of 1
Loading...
Thumbnail Image
Name:
CONE-DOCUMENT-2021.pdf
Size:
7.26 MB
Format:
Adobe Portable Document Format
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.6 KB
Format:
Plain Text
Description: