Photometric Reconstruction Under Complex Imaging Conditions

Traditionally, photometric 3D reconstruction has relied on simple, Lambertian assumptions. In this line of work, we develop novel theoretical frameworks to handle complex phenomena like interreflections, shadows, non-Lambertian BRDF and global light transport. Importantly, we present algorithms that leverage on the theories to undo those non-ideal effects and produce accurate, high quality 3D reconstructions.

Dealing with Complex BRDFs

Complex BRDFs

In this work, we present a comprehensive theoretical and algorithmic analysis of photometric reconstruction for surfaces with complex BRDFs. We derive invariants that relate the shape of a surface to image derivatives, for arbitrary, unknown isotropic BRDFs. The forms of these invariants are strikingly similar to optical flow, but without requiring restrictive brightness constancy assumptions. We also delineate exact topological classes up to which reconstruction is possible for particular lighting and material conditions. For more details, visit the respective pages for light source motion, object motion and camera motion.

Publications:

  1. M.K. Chandraker. What Camera Motion Reveals About Shape with Unknown BRDF, CVPR 2014, Columbus, Ohio. [PDF] [Tech Report] [oral]
  2. M.K. Chandraker, D. Reddy, Y. Wang and R. Ramamoorthi. What Motion Reveals About Shape with Unknown BRDF and Lighting, CVPR 2013, Portland, Oregon. [PDF] [oral]
  3. M.K. Chandraker, J. Bai and R. Ramamoorthi. On Differential Photometric Reconstruction for Unknown, Isotropic BRDFs, IEEE PAMI 35(12):2941-2955, December 2013. [PDF] [Spl. Issue, Best of CVPR 2011]
  4. M.K. Chandraker, J. Bai and R. Ramamoorthi. A Theory of Photometric Reconstruction for Unknown Isotropic Reflectances, CVPR 2011, Colorado Springs, pp. 2505-2512. [PDF] [oral]
[Top] [Home]

Dealing with Inter-reflections

branch and bound

We furnish a theoretical proof that the generalized bas-relief ambiguity of uncalibrated photometric stereo is fully resolved in the presence of inter-reflections. We further show that no other ambiguity exists for general surfaces. This is exploited to recover the full Euclidean structure from photometric stereo images with varying, unknown light source directions.

Publications:

  1. M.K. Chandraker, F. Kahl and D.J. Kriegman. Reflections on the Generalized Bas-Relief Ambiguity. CVPR 2005, San Diego, USA. [PDF] [oral]
[Top] [Home]

Dealing with Shadows

branch and bound branch and bound

This work presents surface reconstruction using photometric stereo, even in the presence of shadows. The algorithm has three novel features. First, a fast, graph cuts based algorithm estimates source visibility at every pixel. Second, constrained normal integration is used to reduce the low frequency bias of integration and produce high quality reconstructions consistent with the shadow maps.

Importantly, the method allows images to be acquired with multiple illuminants, which leads to better surface coverage and improved signal to noise ratio. The number of images can be less than number of sources, so the imaging effort grows sublinearly with number of sources.

Publications:

  1. M.K. Chandraker, S. Agarwal and D.J. Kriegman. ShadowCuts: Photometric Stereo with Shadows. CVPR 2007, Minneapolis, USA. [PDF]
[Top] [Home]

Dealing with Global Light Transport

branch and bound

A cornerstone of computer graphics is the solution of the rendering equation, which allows the simulation of global illumination, given direct lighting or corresponding light source emissions. This paper lays the foundations for the inverse problem, whereby a dual theoretical framework is presented for inverting the rendering equation to undo interreflections in a real scene, thereby obtaining the direct lighting. Physically, we show that each term of our inverse series cancels an interreflection bounce, just as the forward series adds them. In algorithmic terms, we develop the analog of iterative finite element methods like forward radiosity to efficiently solve light transport inversion. Our iterative inverse light transport algorithm is very fast, requiring only matrix-vector multiplications. We also explore the connections to forward rendering in terms of Monte Carlo and wavelet-based techniques. For more details, click here.

Publications:

  1. J. Bai, M.K. Chandraker, T.-T. Ng and R. Ramamoorthi. A Dual Theory of Inverse and Forward Light Transport. ECCV 2010, Heraklion, Greece. [PDF]
  2. M.K. Chandraker, J. Bai, T.-T. Ng and R. Ramamoorthi. On the Duality of Forward and Inverse Light Transport, IEEE PAMI 33(10):2122-2128, October 2011. [PDF]
[Top] [Home]

Last updated May 31, 2014.