Skip navigation.

UCSD Computer Vision

Light Fields for Computer Vision

Ravi Ramamoorthi, Jong-Chyi Su


Synopsis

The recent advent of commercial light field cameras, as well as depth cameras, opens up new opportunities for computer vision, with richer sensors. We are interested in the full stack of problems, from theoretical analysis of light field camera resolution, to shape recovery from multiple light field or RGBZ cues, to graphics applications, to higher-level vision applications in segmentation and recognition.


Non UCSD Vision People also Involved

Michael Tao (UC Berkeley)


Details

An important recent development is the commercial release of light field cameras from Lytro, Raytrix and Pelican. These cameras consider the whole light field rather than a single image. Since they keep the angular information, operations like refocusing after capture and changing one's perspective are also possible. However, light fields (and related depth cameras) have much broader implications in computer vision. The availability of multiple views and additional cues in a single shot passive capture enables new methods for shape recovery. Light field cameras also enable new approaches to higher-level computer vision tasks such as matting and compositing, or segmentation and recognition. Moreover, non-Lambertian reflectance and difficult photometry can be more easily handled. We seek a comprehensive end-to-end understanding from a core analysis of light field camera resolution to light field shape recovery to higher-level computer vision operations.


Related Publications

2015
Liang C., Ramamoorthi R., "A Light Transport Framework for Lenslet Light Field Cameras", ACM Transactions on Graphics (TOG), 2015. [BibTex][pdf]
2014
Tao M., Wang T., Malik J., Ramamoorthi R., "Depth Estimation for Glossy Surfaces with Light-Field Cameras", ECCV Workshop on Light Fields for Computer Vision (L4CV), 2014. [BibTex][pdf]
2013
Tao M., Hadap S., Malik J., Ramamoorthi R., "Depth from Combining Defocus and Correspondence Using Light Field Cameras", International Conference on Computer Vision (ICCV), pp. 673-680, 2013. [BibTex][pdf]
Reddy D., Bai J., Ramamoorthi R., "External Mask-Based Depth and Light Field Camera", ICCV Workshop on Consumer Depth Cameras for Computer Vision, 2013. [BibTex][pdf]
Tao M., Malik J., Ramamoorthi R., "Sharpening Out of Focus Images using High-Frequency Transfer", Computer Graphics Forum, vol. 32, no. 2, pp. 489-498, May, 2013. [www] [BibTex][pdf]
2009
Gu J., Ramamoorthi R., Belhumeur P., Nayar S., "Removing Image Artifacts Due to Dirty Camera Lenses and Thin Occluders", ACM Transactions on Graphics (SIGGRAPH Asia 2009), vol. 28, no. 5, Dec, 2009. [www] [BibTex][pdf]