CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo

PubDate: June 2021

Teams: Rice University 2Facebook Reality Labs

Writers: Shiyu Tan1,* Yicheng Wu1,* Shoou-I Yu2 Ashok Veeraraghavan1,†

PDF: CodedStereo: Learned Phase Masks for Large Depth-of-field Stereo

Abstract

Conventional stereo suffers from a fundamental trade-off between imaging volume and signal-to- noise ratio (SNR) – due to the conflicting impact of aperture size on both these variables. Inspired by the extended depth of field cameras, we propose a novel end-to-end learning-based technique to overcome this limitation, by introducing a phase mask at the aperture plane of the cameras in a stereo imaging system. The phase mask creates a depth-dependent yet numerically invertible point spread function, allowing us to recover sharp image texture and stereo correspondence over a significantly extended depth of field (EDOF) than conventional stereo. The phase mask pattern, the EDOF image reconstruction, and the stereo disparity estimation are all trained together using an end-to-end learned deep neural network. We perform theoretical analysis and characterization of the proposed approach and show a 6× increase in volume that can be imaged in simulation. We also build an experimental prototype and validate the approach using real-world results acquired using this prototype system