Improving the Robustness of Deep Learning Based Image Reconstruction Models Against Natural Distribution Shifts

Simple item page
dc.contributor.advisorHeckel, Reinharden_US
dc.contributor.advisorBaraniuk, Richard G.en_US
dc.creatorZalbagi Darestani, Mohammaden_US
dc.date.accessioned2023-08-09T18:51:35Zen_US
dc.date.available2023-08-09T18:51:35Zen_US
dc.date.created2023-05en_US
dc.date.issued2023-04-14en_US
dc.date.submittedMay 2023en_US
dc.date.updated2023-08-09T18:51:35Zen_US
dc.description.abstractDeep neural networks give state-of-the-art accuracy for reconstructing images from few and noisy measurements, a problem arising for example in accelerated magnetic resonance imaging (MRI). However, recent works have raised concerns that deep-learning-based image reconstruction methods are sensitive to perturbations and are less robust than traditional methods: Neural networks (i) may be sensitive to small, yet adversarially-selected perturbations, (ii) may perform poorly under distribution shifts, and (iii) may fail to recover small but important features in an image. In order to understand the sensitivity to such perturbations, in this work, we measure the robustness of different approaches for image reconstruction including trained and un-trained neural networks as well as traditional sparsity-based methods. We find, contrary to prior works, that both trained and un-trained methods are vulnerable to adversarial perturbations. Moreover, both trained and un-trained methods tuned for a particular dataset suffer very similarly from distribution shifts. Finally, we demonstrate that an image reconstruction method that achieves higher reconstruction quality, also performs better in terms of accurately recovering fine details. Our results indicate that the state-of-the-art deep-learning-based image reconstruction methods provide improved performance than traditional methods without compromising robustness. Based on the afore-mentioned conclusions, we target deep learning based models in order to improve their robustness to natural distribution shifts. Note that this is a natural choice since deep learning based models are equally vulnerable to distribution shifts compared to other families of reconstruction methods, yet they give superior performance. We also selected natural distribution shifts since they occur frequently in practice. For example, we train a network on data from one hospital, and apply the network to data from a different hospital. Or we train on data acquired with one scanner type and acquisition mode, and apply it to a different scanner type or acquisition mode. In this work, we propose a domain adaptation method for deep learning based compressive sensing that relies on self-supervision during training paired with test-time training at inference. We show that for four natural distribution shifts, this method essentially closes the distribution shift performance gap for state-of-the-art architectures for accelerated MRI.en_US
dc.format.mimetypeapplication/pdfen_US
dc.identifier.citationZalbagi Darestani, Mohammad. "Improving the Robustness of Deep Learning Based Image Reconstruction Models Against Natural Distribution Shifts." (2023) Diss., Rice University. <a href="https://hdl.handle.net/1911/115153">https://hdl.handle.net/1911/115153</a>.en_US
dc.identifier.urihttps://hdl.handle.net/1911/115153en_US
dc.language.isoengen_US
dc.rightsCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.en_US
dc.subjectComputer Visionen_US
dc.subjectRobustnessen_US
dc.subjectDeep Learningen_US
dc.subjectImage Reconstructionen_US
dc.subjectCompressed Sensingen_US
dc.titleImproving the Robustness of Deep Learning Based Image Reconstruction Models Against Natural Distribution Shiftsen_US
dc.typeThesisen_US
dc.type.materialTexten_US
thesis.degree.departmentElectrical and Computer Engineeringen_US
thesis.degree.disciplineEngineeringen_US
thesis.degree.grantorRice Universityen_US
thesis.degree.levelDoctoralen_US
thesis.degree.nameDoctor of Philosophyen_US
Files
Original bundle
Now showing 1 - 1 of 1
Thumbnail Image
Name:
ZALBAGIDARESTANI-DOCUMENT-2023.pdf
Size:
4.62 MB
Format:
Adobe Portable Document Format
Download
License bundle
Now showing 1 - 2 of 2
No Thumbnail Available
Name:
PROQUEST_LICENSE.txt
Size:
5.86 KB
Format:
Plain Text
Description:
Download
No Thumbnail Available
Name:
LICENSE.txt
Size:
2.62 KB
Format:
Plain Text
Description:
Download
Collections
Rice University Theses and Dissertations