The Recurrent Neural Tangent Kernel

Simple item page
dc.contributor.advisorBaraniuk, Richard G.
dc.creatorAlemohammad, Sina
dc.date.accessioned2022-10-05T21:27:32Z
dc.date.available2022-10-05T21:27:32Z
dc.date.created2022-05
dc.date.issued2022-05-09
dc.date.submittedMay 2022
dc.date.updated2022-10-05T21:27:32Z
dc.description.abstractThe study of deep neural networks (DNNs) in the infinite-width limit, via the so-called neural tangent kernel (NTK) approach, has provided new insights into the dynamics of learning, generalization, and the impact of initialization. One key DNN architecture remains to be kernelized, namely, the recurrent neural network (RNN). In this thesis we introduce and study the Recurrent Neural Tangent Kernel (RNTK), which provides new insights into the behavior of overparametrized RNNs. A key property of the RNTK should greatly benefit practitioners is its ability to compare inputs of different length. To this end, we characterize how the RNTK weights different time steps to form its output under different initialization parameters and nonlinearity choices. A synthetic and 56 real-world data experiments demonstrate that the RNTK offers significant performance gains over other kernels, including standard NTKs, across a wide array of data sets.
dc.format.mimetypeapplication/pdf
dc.identifier.citationAlemohammad, Sina. "The Recurrent Neural Tangent Kernel." (2022) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/113528">https://hdl.handle.net/1911/113528</a>.
dc.identifier.urihttps://hdl.handle.net/1911/113528
dc.language.isoeng
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
dc.subjectNeural Tangent Kernel
dc.subjectRecurrent Neural Networks
dc.titleThe Recurrent Neural Tangent Kernel
dc.typeThesis
dc.type.materialText
thesis.degree.departmentElectrical and Computer Engineering
thesis.degree.disciplineEngineering
thesis.degree.grantorRice University
thesis.degree.levelMasters
thesis.degree.nameMaster of Science
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
ALEMOHAMMAD-DOCUMENT-2022.pdf
Size:
1.72 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.85 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.61 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Electronic Theses and Dissertations