Material Recovery and Editing Using a Single Image



We derive precise conditions under which material reflectance properties may be estimated from a single image of a homogeneous curved surface (canonically a sphere), lit by a directional source. Based on the observation that light is reflected along certain (a priori unknown) preferred directions such as the half-angle, we propose a semiparametric BRDF abstraction that lies between purely parametric and purely data-driven models. Formulating BRDF estimation as a particular type of semiparametric regression, both the preferred directions and the form of BRDF variation along them can be estimated from data. Our approach has significant theoretical, algorithmic and empirical benefits, lends insights into material behavior and enables novel applications. While it is well-known that fitting multi-lobe BRDFs may be ill-posed under certain conditions, prior to this work, precise results for the well-posedness of BRDF estimation had remained elusive. Since our BRDF representation is derived from physical intuition, but relies on data, we avoid pitfalls of both parametric (low generalizability) and non-parametric regression (low interpretability, curse of dimensionality). Finally, we discuss several applications such as single-image relighting, light source estimation and physically meaningful BRDF editing.


M.K. Chandraker and R. Ramamoorthi [oral]
What An Image Reveals About Material Reflectance [PDF]
ICCV 2011, Barcelona, Spain.

Details and sample applications:

Application: Light source estimation

Both the best projection direction and BRDF variation along it are computed from data. Knowing that a half-angle model is followed, one may use this to estimate the light source direction. The constancy of the curves across lighting directions is used as the basis for the next application: relighting.

photometric reconstruction

Application: Image relighting

Given an input image, the BRDF is estimated. The estimation with a 1-term PPR fits the data better than parametric models. The estimated BRDF can be used to relight both simlpe and complex geometries.

photometric reconstruction

Application: Material editing in images

An advantage of our approach is that both the BRDF directions and functions can be edited simultaneously. Given an input image of a sphere, the 2-lobe BRDF is estimated and a complex geometry is relighted. Both the direction and strength of gloss in the relighting can be edited individually as well as simultaneously.

photometric reconstruction

Last updated May 31, 2014.