Robust computation of optical flow in a multi-scale differential framework

Abstract

The authors developed an algorithm for computing optical flow in a differential framework. The image sequence is first convolved with a set of linear, separable spatiotemporal filters similar to those that have been used in other early vision. The brightness constancy constraint can then be applied to each of the resulting images, giving in general, an overdetermined system of equations for the optical flow at each pixel. There are three principal sources of error: (a) stochastic due to sensor noise, (b) systematic errors in the presence of large displacements, and (c) errors due to failure of the brightness constancy model. The analysis of these errors leads to development of an algorithm based on a robust version of total least squares. Each optical flow vector computed has an associated reliability measure which can be used in subsequent processing. The performance of the algorithm on the data set used by J. Barron et al. (1992) compares favorably with other techniques. In addition to being separable, the filters used are also causal, incorporating only past time frames. The algorithm is fully parallel and has been implemented on a multipleprocessor machine.<>