Overparameterization and double descent in PCA, GANs, and Diffusion models

Simple item page
dc.contributor.advisorBaraniuk, Richard G
dc.creatorLuzi, Lorenzo
dc.date.accessioned2024-05-22T17:34:20Z
dc.date.available2024-05-22T17:34:20Z
dc.date.created2024-05
dc.date.issued2024-04-19
dc.date.submittedMay 2024
dc.date.updated2024-05-22T17:34:20Z
dc.description.abstractThis PhD thesis constitutes a synthesis of my doctoral work, which addresses various aspects of study related to generative modeling with a particular focus on overparameterization. Using a novel method we call pseudo-supervision, we investigate approaches toward characterization of overparameterization behaviors, including double descent, of GANs as well as PCA-like problems. Extending pseudo-supervision to diffusion models, we see that it can be used to create an inductive bias; we demonstrate that this allows us to train our model with lower generalization error and faster convergence time compared to the baseline. I additionally introduce a novel method called Boomerang to extend our study of diffusion models, showing that they can be used for local sampling in image manifolds. Finally, in an approach we titled WaM, I extend FID to include non-Gaussian distributions by using a Gaussian mixture model and a bound on the 2-Wasserstein metric for Gaussian mixture models to define a metric on non-Gaussian features.
dc.format.mimetypeapplication/pdf
dc.identifier.citationLuzi, Lorenzo. Overparameterization and double descent in PCA, GANs, and Diffusion models. (2024). PhD diss., Rice University. https://hdl.handle.net/1911/116219
dc.identifier.urihttps://hdl.handle.net/1911/116219
dc.language.isoeng
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
dc.subjectoverparameterization
dc.subjectgenerative models
dc.subjectgans
dc.subjectdiffusion models
dc.subjectdouble descent
dc.titleOverparameterization and double descent in PCA, GANs, and Diffusion models
dc.typeThesis
dc.type.materialText
thesis.degree.departmentElectrical and Computer Engineering
thesis.degree.disciplineEngineering
thesis.degree.grantorRice University
thesis.degree.levelDoctoral
thesis.degree.nameDoctor of Philosophy
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
LUZI-DOCUMENT-2024.pdf
Size:
7.69 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.84 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.98 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Electronic Theses and Dissertations