The Extraction of Spatio-temporal Energy in Human and Machine Vision

Edward H. Adelson and James R. Bergen

Published in
Proceedings of the Workshop on Motion: Representation and Analysis (pp. 151-155)
Charleston, SC; May 7-9 (1986)

Recent work in human motion perception has conceptualized motion detection in terms of filters selective for spatio-temporal (ST) energy. One class of such models are called "energy models." They code motion energy, not velocity as such, but we describe a velocity coding model based on the ST energy within given bands of spatial and temporal frequency. The motion energy computations appear at first to be quite different from the gradient-based computations that have been used in machine vision. But gradient systems that make appropriate use of confidence can be considered to derive velocity from opponent energy measures. We also note that in human vision, dynamic energy seems to be coded in a set of transformed axes, which are opponent motion energy (e.g. right - left) and counter-phase flicker energy. The flicker axis is often neglected, we note its possible utility in visual processing.