Deep Learning (DL) has revolutionized various applications in computational imaging and computer vision. However, existing DL frameworks are mostly data-driven, which largely disregards decades of prior work that focused on signal processing theory and physics-based models. As a result, many DL based image reconstruction methods generate eye-pleasing results but faces strong drawbacks, including 1) output not being physically correct, 2) requiring large datasets with labor-intensive annotations. In the thesis, we propose several computational imaging frameworks that leverage both physics-based models and data-driven deep learning.

By formulating the physical model as an integrated and differentiable layer of the larger learning networks, we are able to a) constraint the results to be closer to the physical reality, b) perform self-supervised network training using the physical constraints as loss functions, avoiding manually labeled data, and c) develop true end-to-end imaging systems with jointly optimized front-end sensors and back-end algorithms. In particular, we show that the proposed approach is suitable for a wide range of applications, including motion de-blurring, 3D imaging and super-resolution microscopy.