Contrastive Learning in Deep Learning
| dc.contributor.advisor | Kyrillidis, Anastasios | en_US |
| dc.creator | Chen, John | en_US |
| dc.date.accessioned | 2024-01-24T21:40:48Z | en_US |
| dc.date.available | 2024-01-24T21:40:48Z | en_US |
| dc.date.created | 2023-12 | en_US |
| dc.date.issued | 2023-10-12 | en_US |
| dc.date.submitted | December 2023 | en_US |
| dc.date.updated | 2024-01-24T21:40:48Z | en_US |
| dc.description.abstract | Contrastive Learning is a popular method for training modern deep neural networks. In this thesis, we explore several methods in the supervised learning and semi-supervised learning setting. Firstly, we propose a technique called Negative Sampling in Semi-Supervised Learning (NS3L). NS3L exploits implicit negative evidence to improve the top-line performance of deep neural networks in semi-supervised learning. NS3L requires almost no additional computation and overhead and is shown to improve existing state-of-the-art methods. Secondly, we take the view of implicit contrastive learning and propose the data augmentation method StackMix. Following the “Mix” line of work, StackMix takes pairs of samples and concatenates the inputs while averaging the outputs. This way, the neural network needs to learn to differentiate between the two samples within the concatenated sample. Improved performance is demonstrated on a variety of settings. Lastly, we tackle the computational requirements of FixMatch, a semi-supervised learning method, and propose Fast FixMatch based on curriculum batch size. Curriculum batch size exploits natural training dynamics by starting with a small batch size and ending with a large batch size. Coupled with two other complementary methods that together perform better than a sum of parts, Fast FixMatch demonstrates substantial decreased training computations compared with FixMatch. | en_US |
| dc.format.mimetype | application/pdf | en_US |
| dc.identifier.citation | Chen, John. "Contrastive Learning in Deep Learning." (2023). PhD diss., Rice University. https://hdl.handle.net/1911/115392 | en_US |
| dc.identifier.uri | https://hdl.handle.net/1911/115392 | en_US |
| dc.language.iso | eng | en_US |
| dc.rights | Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder. | en_US |
| dc.subject | Contrastive Learning | en_US |
| dc.subject | Deep Learning | en_US |
| dc.subject | Computer Vision | en_US |
| dc.title | Contrastive Learning in Deep Learning | en_US |
| dc.type | Thesis | en_US |
| dc.type.material | Text | en_US |
| thesis.degree.department | Computer Science | en_US |
| thesis.degree.discipline | Engineering | en_US |
| thesis.degree.grantor | Rice University | en_US |
| thesis.degree.level | Doctoral | en_US |
| thesis.degree.name | Doctor of Philosophy | en_US |
Files
Original bundle
1 - 1 of 1