Plenoptic Camera

Plenoptic Camera

This Demo is created by John Y. A. Wang


The Plenoptic camera is a novel single-lens camera that captures 3-D scene information. A plenoptic image is captured with a single lens and a lenticular lens array at the imaging plane. This image, like an insect's compound eye, consists of multiple sub-images each imaging the scene in front of it, in the plenoptic camera, the main lens aperture is imaged behind each lenticular element.

The image behind the lenticular array is digitized to obtain a digital plenoptic image. As a result, each sub-image is represented by a localized array of pixels that encodes the path of light rays entering the main aperture. For a large aperture, various pixels of each sub-image will be lit in the sub-image. For a small aperture, only one pixel will be lit.

Camera Optics This figure illustrate how views from the top, center, and bottom of the camera aperture can be obtained by electroniclly selecting the appropriate pixels from the plenoptic image.

Consider a camera with an aperture that activate only one pixel behind each lenticular element. As this aperture is translated, it activate different pixels. For example, forming an image from the lower pixels correspond to a top view. Likewise, forming images from the central pixels or the upper pixels correspond to center and bottom views, respectively.

Thus by using a wide open aperture, all views are captured simulatneously by the plenoptic image without any movement of the aperture or camera components. The optics of the plenoptic camera and the arrangement of pixels allow these views to be electronically decoded for recovery of 3D scene structure.

Details of the optics can be better understood with a movie ( mov, mov.gz ) illustrating the mapping of the pixels to the effective aperture.

The plenoptic camera is self-calibrating and requires only a single-lens camera. It provides views along both the vertical and horizontal axis to recover structure of any orientation. Furthermore the novel optics eliminates spatial aliasing, correspondence and aperture problems, and temporal aliasing problems present in conventional stereo and structure from motion.


E. H. Adelson and J. Y. A. Wang. Single Lens Stereo with Plenoptic Camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 14, No. 2, pp. 99-106, February 1992.

E. H. Adelson and J. Y. A. Wang. A Stereoscopic Camera Employing a Single Main Lens. Proceedings IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1991, Hawaii, June 1991, pp. 619-624.


Adelson, Optical Ranging Aparatus. US Patent #05076687.

Example of 3D Shape recovery of Lego(tm) Pyramid

Lego pyramid

Raw images captured by the plenoptic camera

Plenoptic image
A plenoptic image of Lego pyramid. The array of bright spots indicates the structure of the lenticular array. (Note the image of the pyramid is barely visible)
Zoomed in image
Zoomed in view of the plenoptic image showing pixel distribution behind the lenticular elements.

Views along the vertical and horizontal axes

. . view[1][3] ..
.. view[2][3] ..
view[3][1] view[3][2] view[3][3] view[3][4] view[3][5]
.. view[4][3] ..
.. view[5][3] ..

In this example, we obatined a 5x5 array of images by electronic decomposition of a single plenoptic image. Show here are the images corresponding to views that would be seen along the vertical and horizontal axes of the camera aperture.

Movies made from these images dramtically show the effects of parallax.

3D Recovery

Color range map

This color range map shows three distinct depth levels corresponding to Lego pyramid obtained by processing 25 views captured by the plenoptic camera. The blue colors indicate points farther from the camera and the red colors indicate points closer to the camera. The color bar show their relative distances.

MIT BCS Perceptual Science Group. Demo by John Y. A. Wang. <>
Copyright 1995. All rights reserved.

Last modified: Thu Jul 24 22:19:15 1997