Stereo Reconstruction for Complex, Unknown Material Type

camera motion

Summary:

Psychophysical studies show motion cues inform about shape even with unknown reflectance. Recent works in computer vision have considered shape recovery for an object of unknown BRDF using light source or object motions. This paper addresses the remaining problem of determining shape from the (small or differential) motion of the camera, for unknown isotropic BRDFs. Our theory derives a differential stereo relation that relates camera motion to depth of a surface with unknown isotropic BRDF, which generalizes traditional Lambertian assumptions. Under orthographic projection, we show shape may not be constrained in general, but two motions suffice to yield an invariant for several restricted (still unknown) BRDFs exhibited by common materials. For the perspective case, we show that three differential motions suffice to yield surface depth for unknown isotropic BRDF and unknown directional lighting, while additional constraints are obtained with restrictions on BRDF or lighting. The limits imposed by our theory are intrinsic to the shape recovery problem and independent of choice of reconstruction method. We outline with experiments how potential reconstruction methods may exploit our theory. We illustrate trends shared by theories on shape from motion of light, object or camera, relating reconstruction hardness to imaging complexity.

Publications:

M.K. Chandraker [oral]
What Camera Motion Reveals About Shape with Unknown BRDF [PDF] [Tech Report]
CVPR 2014, Columbus, Ohio.


Details and sample results:

Stratification of surface recovery from camera motion:

photometric reconstruction

Unified framework for shape recovery from light, object or camera motions:

photometric reconstruction

Shape recovery under orthographic and perspective projections:

photometric reconstruction

Last updated May 31, 2014.