Third-Order Interactions in Neural Computations

dc.contributor.advisorHafner, Jason H.
dc.contributor.advisorPitkow, Xaq
dc.creatorFei, Yicheng
dc.date.accessioned2024-05-21T15:24:33Z
dc.date.created2024-05
dc.date.issued2024-04-17
dc.date.submittedMay 2024
dc.date.updated2024-05-21T15:24:33Z
dc.descriptionEMBARGO NOTE: This item is embargoed until 2024-11-01
dc.description.abstractIn this thesis, we explore the role of third-order interactions in neural computations, emphasizing their significance as a reflection of such generative processes in the physical world. We also propose using third-order interactions in probabilistic graphical models (PGMs) within the exponential family as a normative way to define a gating mechanism in generative probabilistic graphical models. By going a step beyond pairwise interactions, it empowers much more computational efficiency, like a transistor expands possible digital computations. We also demonstrate the use of third-order PGM for explaining observed properties of neural computations, particularly in context-dependent flexible divisive normalization and attention. Both can be conceptualized as a gating mechanism. As a concrete example, we show that a graphical model with three-way interactions provides a normative explanation for observed divisive normalization properties in the macaque primary visual cortex. Inference in such PGMs is nontrivial. We define Recurrent Factor Graph Neural Network (RF-GNN), a machine learning approach developed for fast approximate inference in PGMs with higher-order interactions. Experimental results on several families of graphical models demonstrate the out-of-distribution generalization capability of our method to different-sized graphs and indicate the domain in which our method outperforms Belief Propagation (BP). Moreover, we test the RF-GNN on a real-world Low-Density Parity-Check dataset as a benchmark along with other baseline models including BP variants and a stacked GNN method. Overall we find that RF-GNNs outperform other methods under high noise levels.
dc.embargo.lift2024-11-01
dc.embargo.terms2024-11-01
dc.format.mimetypeapplication/pdf
dc.identifier.citationFei, Yicheng. Third-Order Interactions in Neural Computations. (2024). PhD diss., Rice University. https://hdl.handle.net/1911/116028
dc.identifier.urihttps://hdl.handle.net/1911/116028
dc.language.isoeng
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
dc.subjectHigher-order interaction
dc.subjectGraph neural network
dc.subjectDivisive normalization
dc.subjectAttention
dc.subjectProbabilistic graphical model
dc.titleThird-Order Interactions in Neural Computations
dc.typeThesis
dc.type.materialText
thesis.degree.departmentPhysics and Astronomy
thesis.degree.disciplineNatural Sciences
thesis.degree.grantorRice University
thesis.degree.levelDoctoral
thesis.degree.nameDoctor of Philosophy
Files
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.98 KB
Format:
Plain Text
Description: