Optical Flow for Unknown Material Types

optical flow


We present a theory that addresses the problem of determining shape from the (small or differential) motion of an object with unknown isotropic reflectance, under arbitrary unknown distant illumination, for both orthographic and perpsective projection. Our theory imposes fundamental limits on the hardness of surface reconstruction, independent of the method involved. Under orthographic projection, we prove that three differential motions suffice to yield an invariant that relates shape to image derivatives, regardless of BRDF and illumination. Under perspective projection, we show that four differential motions suffice to yield depth and a linear constraint on the surface gradient, with unknown BRDF and lighting. Further, we delineate the topological classes up to which reconstruction may be achieved using the invariants. Finally, we derive a general stratification that relates hardness of shape recovery to scene complexity. Qualitatively, our invariants are homogeneous partial differential equations for simple lighting and inhomogeneous for complex illumination. Quantitatively, our framework shows that the minimal number of motions required to resolve shape is greater for more complex scenes. Prior works that assume brightness constancy, Lambertian BRDF or a known directional light source follow as special cases of our stratification. We illustrate with synthetic and real data how potential reconstruction methods may exploit our framework.


M.K. Chandraker, D. Reddy, Y. Wang and R. Ramamoorthi [oral]
What Motion Reveals About Shape with Unknown BRDF and Lighting [PDF]
CVPR 2013, Portland, Oregon.

Details and sample results:

Stratification of surface recovery from object motion:

photometric reconstruction

Shape recovery under orthographic projection:

photometric reconstruction

Shape recovery under perspective projection:

photometric reconstruction

Last updated May 31, 2014.