Snapshot Compressive ToF+Spectral Imaging via Optimized Color-Coded Apertures.

Compressive-multispectral-imaging systems comprise a new generation of spectral imagers that capture coded projections of a scene where spectral data-cubes are reconstructed computationally. Separately, time-of-flight (ToF) cameras obtain 2D range images where each pixel records the distance from the camera-sensor to the target-surface. The demand for these imaging modalities is rapidly increasing, and thus, there is strong interest in developing new image-sensors that can simultaneously acquire multispectral-color-and-depth imagery (MS+D) using a single-aperture. Work in this path has been mainly developed via RGB+D imaging. However, in RGB+D, the multispectral image is limited to three spectral channels, and the imaging system often relies on two image-sensors. We recently proposed a compressive MS+D imaging device that used a digital-micromirror-device, requiring a bulky double imaging-and-relay path. To overcome the bulkiness and other difficulties of our previous imaging system, this work presents a more-compact MS+D imaging device with snapshot capabilities. It provides better spectral sensing, relying on a static color-coded-aperture (CCA) and a ToF sensor. To guarantee good quality in the recovery, we develop an optimization method for CCA based-on blue-noise-multitoning, solved via the direct-binary-search algorithm. A testbed-setup is reported along with simulated-and-real experiments that demonstrate the MS+D capabilities of the proposed system over static-and-dynamic scenes.