|
My primary research interest is developing mathematical approaches to solving 3D reconstruction problems in computer vision. In particular, I like developing continuous and discrete optimization algorithms with nice theoretical properties. Some of my work in geometric reconstruction presents globally optimal solutions to several problems in multiview geometry. Recently, I have also developed a real-time structure-from-motion system for Honda's humanoid robot, ASIMO. My work in photometric reconstruction develops theories and algorithms for shape recovery in the presence of complex effects like inter-reflections, shadows, non-Lambertian BRDFs and global light transport. |
 |
 |
In this line of work, we present globally optimal solutions to several important problems in multiview geometry. Structure from motion problems are inherently non-convex and these works rely on efficient construction of relaxations using developments in convex optimization and algebraic geometry. |
Globally Optimal Stratified Autocalibration |
 |
We present a practical, stratified autocalibration algorithm with theoretical guarantees of global optimality. Given a projective reconstruction, the first stage of the algorithm upgrades it to affine by estimating the globally optimal position of the plane at infinity. In the second stage, a metric upgrade is performed by computing the globally optimal dual image of the absolute conic (DIAC). For each stage, we construct and minimize tight convex relaxations of the highly non-convex objective functions in a branch and bound optimization framework. We exploit the problem structure to restrict the search space for the DIAC and the plane at infinity to a small, fixed number of branching dimensions, independent of the number of views. |
Publications :
|
Direct Autocalibration |
 |
 |
Direct autocalibration recovers the metric reconstruction in a single step, by estimating the absolute dual quadric. The problem is formulated as a polynomial objective function, subject to polynomial constraints. Rather than use locally optimal non-linear minimization, we construct convex linear matrix inequality relaxations of the polynomial program to globally minimize it. Unlike prior work, our framework enforces critical requirements like rank degeneracy and semidefiniteness of the quadric within the estimation, rather than as post-processing. |
Publications :
|
Optimal Bilinear Programming |
 |
 |
A wide variety of problems in computer vision can be posed as bilinear programs, for which we provide a globally optimal solution, in both standard L2 and robust L1 norms. Efficient convex elaxations are constructed and minimized in a branch and bound paradigm. The main theoretical result proves that the structure of common bilinear programs in computer vision allows a drastic reduction in search space. Example applications include 3D face reconstruction from exemplars using a single input image and non-rigid structure-from-motion. |
Publications :
|
Optimal Triangulation and Camera Resection |
 |
 |
We present globally optimal solutions to triangulation and camera resectioning, minimizing optimal reprojection error in the standard L2-norm as well as the robust L1-norm. We show these problems to be instances of a wider class of fractional programs, for which we construct efficient second order cone programming relaxations. Subsequently, global minimization is carried out in a branch and bound framework, exploiting the properties of multiview geometry to achieve rapid convergence in practice. |
Publications :
|
Traditionally, photometric 3D reconstruction has relied on simple, Lambertian assumptions. In this line of work, we elucidate the theory of complex phenomena like interreflections, shadows, non-Lambertian BRDF and global light transport. Importantly, we present algorithms that leverage on the theory to undo these non-ideal effects and produce accurate 3D reconstructions.
Dealing with Inter-reflections |
 |
We furnish a theoretical proof that the generalized bas-relief ambiguity of uncalibrated photometric stereo is fully resolved in the presence of inter-reflections. We further show that no other ambiguity exists for general surfaces. This is exploited to recover the full Euclidean structure from photometric stereo images with varying, unknown light source directions. |
Publications :
|
Dealing with Shadows |
 |
 |
This work presents surface reconstruction using photometric stereo, even in the presence of shadows. The algorithm has three novel features. First, a fast, graph cuts based algorithm estimates source visibility at every pixel. Second, constrained normal integration is used to reduce the low frequency bias of integration and produce high quality reconstructions consistent with the shadow maps. Importantly, the method allows images to be acquired with multiple illuminants, which leads to better surface coverage and improved signal to noise ratio. The number of images can be less than number of sources, so the imaging effort grows sublinearly with number of sources. |
Publications :
|
Dealing with Global Light Transport |
 |
A cornerstone of computer graphics is the solution of the rendering equation, which allows the simulation of global illumination, given direct lighting or corresponding light source emissions. This paper lays the foundations for the inverse problem, whereby a dual theoretical framework is presented for inverting the rendering equation to undo interreflections in a real scene, thereby obtaining the direct lighting. Physically, we show that each term of our inverse series cancels an interreflection bounce, just as the forward series adds them. In algorithmic terms, we develop the analog of iterative finite element methods like forward radiosity to efficiently solve light transport inversion. Our iterative inverse light transport algorithm is very fast, requiring only matrix-vector multiplications. We also explore the connections to forward rendering in terms of Monte Carlo and wavelet-based techniques. |
Publications :
|
Dealing with Complex BRDFs |
 |
In this work, we present a comprehensive theoretical and algorithmic analysis of photometric reconstruction for surfaces with complex BRDFs. We derive invariants that relate the shape of a surface to image derivatives, for arbitrary, unknown isotropic BRDFs. This is akin to optical flow for source motion, but without requiring restrictive brightness constancy assumptions. We also delineate exact topological classes up to which reconstruction is possible for particular lighting and material conditions. |
Publications :
|
 |
 |
This work develops the real-time SFM system for localizing Honda's advanced humanoid robot, ASIMO. Input images are obtained through stereo cameras and our principal features of interest for indoor environments are infinite lines, which avoids the drawbacks of unstable end-points. The minimal SFM problem requires only two lines and we present a solution to the resulting overdetermined polynomial system which is fast enough to be used in a RANSAC framework. Our entire pipeline for line detection, tracking, minimal SFM and bundle adjustment is expected to achieve real-time rates in the near future. |
 |
 |
The goal of this project at Microsoft Research, Cambridge, was to enable multi-touch sensitive interactions between a user's hand and a tablet PC. The computer vision challenge was high precision determination of contact with a horizontal surface using overhead stereo cameras, which was solved using a machine learning approach. The project has now evolved into Microsoft's C-Slate. |
 |
 |
My first computer vision project, where we developed a fully-mobile, vision-based indoor tracking system at the Technical University of Graz, Austria. A novel subpixel corner detector contributed to accurate pose estimation and high frame rates were obtained by CMOS image acquisition of small subwindows. |
 |
 |
A multistage approach to gray-level corner detection is proposed in this paper, which is based on fast corner extraction using the Plessey corner detector combined with CMOS image acquisition technologies and localization refinement using a spatial subpixel analysis approach. In comparison to the standard Plessey detector, which can result in localization errors of several pixels, experimental results show an average error of only 0.36 pixels for our algorithm. |
Last updated November 10, 2010.