Layered Representation for Motion Analysis

John Y. A. Wang and Edward H. Adelson

Published in
Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (pp. 361-366)
New York; June (1993).

Standard approaches to motion analysis assume that the optic flow is smooth; such techniques have trouble dealing with occlusion boundaries. The most popular solution is to allow discontinuities in the flow field, imposing the smoothness constraint in a piecewise fashion. But there is a sense in which the discontinuities in flow are artifactual, resulting from the attempt to capture the motion of multiple overlapping objects in a single flow field. Instead we can decompose the image sequence into a set of overlapping layers, where each layer's motion is described by a smooth flow field. The discontinuities in the description are then attributed to object opacities rather than to the flow itself, mirroring the structure of the scene. We have devised a set of techniques for segmenting images into coherently moving regions using affine motion analysis and clustering techniques. We are able to decompose an image into a set of layers along with information about occlusion and depth ordering. We have applied the techniques to the ``flower garden'' sequence. We can analyze the scene into four layers, and then represent the entire 30-frame sequence with a single image of each layer, along with associated motion parameters.