Mosaic based representations of video sequences and their applications

Abstract

Recently, there has been a growing interest in the use of mosaic images to represent the information contained in video sequences. The paper systematically investigates how to go beyond thinking of the mosaic simply as a visualization device, but rather as a basis for efficient representation of video sequences. We describe two different types of mosaics called the static and the dynamic mosaic that are suitable for different needs and scenarios. We discuss a series of extensions to these basic mosaics to provide representations at multiple spatial and temporal resolutions and to handle 3D scene information. We describe techniques for the basic elements of the mosaic construction process, namely alignment, integration, and residual analysis. We describe several applications of mosaic representations including video compression, enhancement, enhanced visualization, and other applications in video indexing, search, and manipulation.