Rendering and Imaging with Transients: A Study of Computational Photography with Scattered Photons

Date
2019-04-19
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract

Imaging or scene reconstruction with directly reflected photons is almost a solved problem. However, in scenarios such as imaging through scattering medium or non-line-of-sight (NLOS) imaging, very few to no directly reflected photons reach the detector. Therefore, it is imperative to build imaging systems that reconstruct the scene with scattered photons. But scattered photons destroy high-frequency content and suffer from low photon count making imaging with scattered photons highly challenging. Recent developments in Lidar technology and computational imaging give us hope to address these challenges. Silicon-based single-photon sensitive Lidars known as single-photon avalanche diodes (SPADs) can detect the scattered photons with high sensitivity and also bin these scattered photons based on their time-of-travel, thereby preserving the high-frequency scene content. Imaging with scattered photons finds applications in robotics, automobile navigation, non-invasive imaging through skin and tissue, cancer diagnostics, and remote sensing.

While imaging through scattering medium has received a few decades of attention, NLOS imaging is gaining attention only in the past few years due to the availability of relatively inexpensive transient detectors such as SPADs and increasing applicability. Several techniques have been developed in the past decade for looking around the corner by exploiting transients (round-trip time of travel of photons). Typically, these techniques are limited by the following five challenges, not necessarily in the same order: (1) Low signal to background ratio (SBR) due to background photons from the clutter. (2) Collection of a large number of measurements typically by illuminating (virtual sources) and imaging (virtual detectors) multiple points on the visible portion of the scene. (3) Post-processing of this large data by a reconstruction algorithm to estimate the hidden scene. (4) Limitations imposed by the current SPAD hardware including low temporal resolution and quantum efficiency. (5) Model mismatch.

To mitigate some of the challenges discussed above, I propose a scanning-based non-line-of-sight imaging technique inspired by the confocal microscopy technique. The key idea is that if the virtual sources (pulsed sources) on the wall are delayed using a quadratic delay profile (much like the quadratic phase of a focusing lens), then these pulses arrive at the same instant at a single point in the hidden volume -- the point being scanned. On the imaging side, applying quadratic delays to the virtual detectors before integration on a single gated detector allows us to ' temporally focus' and scan each point in the hidden volume. By changing the quadratic delay profiles, we can focus light at different points in the hidden volume. The proposed technique (temporal focusing) is more robust to clutter as we scan only the voxels of interest. The acquisition time required is proportional to the size of the hidden volume, and hence, when the region of interest in the hidden volume is small and limited, temporal focusing decreases the acquisition time. Temporal focusing directly images voxels around the corner and hence, does not require any post-processing reconstruction algorithms. However, current hardware limitations also limit the resolution of temporal focusing. While this thesis does not improve the transient imaging hardware, which is advancing thanks to advances in semiconductors and Lidars, I propose to decouple the challenges imposed by current hardware with the algorithm performance with a time-of-flight simulator.

While existing time-of-flight simulators are adequate for certain tasks, such as simulating transient cameras, they are very inefficient for simulating time-gated cameras, such as temporal focusing camera, because of the large number of wasted path samples. I take first steps towards addressing these deficiencies, by introducing a procedure for efficiently sampling paths with a predetermined length, and incorporating it within rendering frameworks tailored towards simulating time-gated imaging. Using an open-sourced implementation of the above, I demonstrate improved rendering performance in a variety of applications, including simulating proximity sensors, imaging through occlusions, depth-selective cameras, transient imaging in dynamic scenes, and finally temporal focusing. After developing the simulator for temporal focusing, I demonstrate the advantages of temporal focusing and characterize the effect of various system parameters using the simulator. Finally, I built a temporal focusing prototype with SPADs to demonstrate the real-world feasibility of the ideas proposed in this thesis.

The current thesis shows a case of imaging with scattered photons for non-line-of-sight imaging. Translating the temporal focusing ideas proposed in this thesis for imaging deep through a tissue, improving the limited transient resolution of the current hardware, solving model mismatch are still open problems.

Description
Degree
Doctor of Philosophy
Type
Thesis
Keywords
Imaging Through Scattering Media, Non-Line-of-Sight Imaging, Lidar, Time-of-Flight Cameras, Physics-Based Rendering, Scanning Non-Line-of-Sight
Citation

Pediredla, Adithya K. "Rendering and Imaging with Transients: A Study of Computational Photography with Scattered Photons." (2019) Diss., Rice University. https://hdl.handle.net/1911/105936.

Has part(s)
Forms part of
Published Version
Rights
Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.
Link to license
Citable link to this page