Object-based video coding using forward-tracking 2D mesh layers

Abstract
This paper proposes mesh-based methods for object-based video coding in a similar context as MPEG-4. The mesh-based approach is aimed at supporting a wide variety of applications for object-based bitstream manipulation and editing. We combine recent mesh-based motion models with the object-based video representation of MPEG-4. A mesh tracking algorithm is developed that enables the tracking of foreground video objects, through forward motion estimation, piecewise affine warping and dynamic mesh updates. Techniques to compress the mesh node positions and motion vectors are discussed. The proposed methods were tested on MPEG test sequences of varying complexity. We have replaced block-based motion compensation of the MPEG-4 video verification model with mesh-based tracking and motion compensation and evaluated the performance of the resulting object-based coder.