CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras
Conference on Computer Vision and Pattern Recognition (CVPR) — 2024
Point-spread-function (PSF) engineering is a well-established computational imaging technique that uses phase masks and other optical elements to embed extra information (e.g., depth) into the images captured by conventional CMOS image sensors. To date, however, PSF-engineering has not been applied to neuromorphic event cameras; a powerful new image sensing technology that responds to changes in the log-intensity of light. This paper establishes theoretical limits (Cramér Rao bounds) on 3D point localization and tracking with PSF-engineered event cameras. Using these bounds, we first demonstrate that existing Fisher phase masks are already near-optimal for localizing static flashing point sources (e.g., blinking fluorescent molecules). We then demonstrate that existing designs are sub-optimal for tracking moving point sources and proceed to use our theory to design optimal phase masks and binary amplitude masks for this task. To overcome the non-convexity of the design problem, we leverage novel implicit neural representation based parameterizations of the phase and amplitude masks. We demonstrate the efficacy of our designs through extensive simulations. We also validate our method with a simple prototype.
@article{ Shah2024CodedEvents,
author = { Shah, Sachin and Chan, Matthew A. and Cai, Haoming and Chen, Jingxi and Kulshrestha, Sakshum and Singh, Chahat Deep and Aloimonos, Yiannis and Metzler, Christopher A. },
title = { CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras },
journal = { Conference on Computer Vision and Pattern Recognition (CVPR) },
year = { 2024 },
}