Overparameterization and double descent in PCA, GANs, and Diffusion models
| dc.contributor.advisor | Baraniuk, Richard G | en_US |
| dc.creator | Luzi, Lorenzo | en_US |
| dc.date.accessioned | 2024-05-22T17:34:20Z | en_US |
| dc.date.available | 2024-05-22T17:34:20Z | en_US |
| dc.date.created | 2024-05 | en_US |
| dc.date.issued | 2024-04-19 | en_US |
| dc.date.submitted | May 2024 | en_US |
| dc.date.updated | 2024-05-22T17:34:20Z | en_US |
| dc.description.abstract | This PhD thesis constitutes a synthesis of my doctoral work, which addresses various aspects of study related to generative modeling with a particular focus on overparameterization. Using a novel method we call pseudo-supervision, we investigate approaches toward characterization of overparameterization behaviors, including double descent, of GANs as well as PCA-like problems. Extending pseudo-supervision to diffusion models, we see that it can be used to create an inductive bias; we demonstrate that this allows us to train our model with lower generalization error and faster convergence time compared to the baseline. I additionally introduce a novel method called Boomerang to extend our study of diffusion models, showing that they can be used for local sampling in image manifolds. Finally, in an approach we titled WaM, I extend FID to include non-Gaussian distributions by using a Gaussian mixture model and a bound on the 2-Wasserstein metric for Gaussian mixture models to define a metric on non-Gaussian features. | en_US |
| dc.format.mimetype | application/pdf | en_US |
| dc.identifier.citation | Luzi, Lorenzo. Overparameterization and double descent in PCA, GANs, and Diffusion models. (2024). PhD diss., Rice University. https://hdl.handle.net/1911/116219 | en_US |
| dc.identifier.uri | https://hdl.handle.net/1911/116219 | en_US |
| dc.language.iso | eng | en_US |
| dc.rights | Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder. | en_US |
| dc.subject | overparameterization | en_US |
| dc.subject | generative models | en_US |
| dc.subject | gans | en_US |
| dc.subject | diffusion models | en_US |
| dc.subject | double descent | en_US |
| dc.title | Overparameterization and double descent in PCA, GANs, and Diffusion models | en_US |
| dc.type | Thesis | en_US |
| dc.type.material | Text | en_US |
| thesis.degree.department | Electrical and Computer Engineering | en_US |
| thesis.degree.discipline | Engineering | en_US |
| thesis.degree.grantor | Rice University | en_US |
| thesis.degree.level | Doctoral | en_US |
| thesis.degree.name | Doctor of Philosophy | en_US |
Files
Original bundle
1 - 1 of 1