2-D mesh tracking for synthetic transfiguration

Abstract

Synthetic object transfiguration is the problem of replacing an existing object in an image sequence with a new one via digital post-processing. Successful transfiguration requires accurate tracking of both motion and intensity variations of the existing object so that the new object can be rendered in exactly the same way. We propose a 2-D mesh tracking method that estimates, on a local basis, the motion and intensity variations of a region of interest. The proposed method is capable of accounting for illumination changes, tracking image regions corresponding to scene objects with non-planar and/or mildly deforming surfaces, and is shown to be effective in real-life situations.