Browsing by Author "Chen, George"

Now showing 1 - 3 of 3
  • Thumbnail Image
    Item
    Efficient Machine Vision Using Computational Cameras
    (2016-10-21) Chen, George; Veeraraghavan, Ashok; Patel, Ankit
    Computational cameras, powered by novel optics and advanced signal processing algorithms, has emerged as a powerful imaging tool that brings orders of magni- tude performance improvements over current camera technology. However, existing computer vision pipelines are still built around conventional digital cameras. In this thesis, we propose a novel computer vision framework that integrates computational cameras for machine vision applications. I explore two possible ways of improving the energy-efficiency and cost-effectiveness under such proposed framework. We first introduce ASP Vision, a jointly designed sensor + deep learning system for visual recognition tasks. ASP Vision utilizes angle sensitive pixels (ASP) to optically compute the first layer of convolutional neural networks (CNN), resulting 10x savings in sensing energy and bandwidth, and 2-4% savings in CNN FLOPs, while achieving similar performance compared to traditional deep learning pipelines. We then present FPA-CS, a focal plane array based compressive sensing architecture that provides a 15x cost savings in high-resolution shortwave infrared (SWIR) video acquisition.
  • Thumbnail Image
    Item
    Leveraging Physics-based Models in Data-driven Computational Imaging
    (2019-04-19) Chen, George; Veeraraghavan, Ashok; Baraniuk, Richard; Shrivastava, Anshumali
    Deep Learning (DL) has revolutionized various applications in computational imaging and computer vision. However, existing DL frameworks are mostly data-driven, which largely disregards decades of prior work that focused on signal processing theory and physics-based models. As a result, many DL based image reconstruction methods generate eye-pleasing results but faces strong drawbacks, including 1) output not being physically correct, 2) requiring large datasets with labor-intensive annotations. In the thesis, we propose several computational imaging frameworks that leverage both physics-based models and data-driven deep learning. By formulating the physical model as an integrated and differentiable layer of the larger learning networks, we are able to a) constraint the results to be closer to the physical reality, b) perform self-supervised network training using the physical constraints as loss functions, avoiding manually labeled data, and c) develop true end-to-end imaging systems with jointly optimized front-end sensors and back-end algorithms. In particular, we show that the proposed approach is suitable for a wide range of applications, including motion de-blurring, 3D imaging and super-resolution microscopy.
  • Thumbnail Image
    Item
    Leveraging Physics-based Models in Data-driven Computational Imaging
    (2019-04-19) Chen, George; Veeraraghavan, Ashok; Baraniuk, Richard; Shrivastava, Anshumali
    Deep Learning (DL) has revolutionized various applications in computational imaging and computer vision. However, existing DL frameworks are mostly data-driven, which largely disregards decades of prior work that focused on signal processing theory and physics-based models. As a result, many DL based image reconstruction methods generate eye-pleasing results but faces strong drawbacks, including 1) output not being physically correct, 2) requiring large datasets with labor-intensive annotations. In the thesis, we propose several computational imaging frameworks that leverage both physics-based models and data-driven deep learning. By formulating the physical model as an integrated and differentiable layer of the larger learning networks, we are able to a) constraint the results to be closer to the physical reality, b) perform self-supervised network training using the physical constraints as loss functions, avoiding manually labeled data, and c) develop true end-to-end imaging systems with jointly optimized front-end sensors and back-end algorithms. In particular, we show that the proposed approach is suitable for a wide range of applications, including motion de-blurring, 3D imaging and super-resolution microscopy.