Multiple Description Coding of Visual Information

Abstract

Nowadays, image and video compression are well developed fields of signal processing. Modern state-of-the-art coders allow better compression with better quality. A new field in signal processing is representation of 3D scenes. 3D visual scenes may be captured by stereoscopic or multi-view camera settings. The captured multi-view video can be compressed directly or converted to more abstract 3D data representations such as 3D dynamic meshes (mesh sequences) and efficiently compressed. In any case, efficiently compressed visual data has to be transmitted over communication channels, such as wireless channels or best-effort networks. This raises the problem of error protection, since most of these channels are error-prone.A common approach to error protection is to consider it as a pure channel problem, separate from the source compression problem. This approach is based on Shannon's work, which states that in principle source and channel coding tasks can be carried out independently with no loss of efficiency. However, this cannot be achieved in practice due to delay requirements and other problems. As an alternative, one can tolerate channel losses. Assuming that not all the data sent has reached the decoder, one can concentrate on ensuring efficient decoding of the correctly received data only. One way to achieve this is to use multiple description coding (MDC). The source is encoded into several descriptions, which are sent to the decoder independently over different channels. The decoder can reconstruct the source with lower yet acceptable quality from any description received. Better reconstruction quality is obtained from more descriptions.This thesis investigates MDC of images, video, stereoscopic video, and 3D meshes thus validating MDC as an error resilience tool for various types of multimedia data. The thesis consists of four main chapters.Chapter 2 deals with MDC of images. It introduces an MDC algorithm based on a 2-stage compression scheme employing B-spline-based image resizing which is used to split the image into the coarse and residual parts. The coarse part is included in both descriptions while the residual part is split into two parts. A bit allocation algorithm optimizes the scheme for a given bit budget and probability of description loss.Chapter 3 addresses MDC of video. It presents a 3D-transform-based MD video coder targeted for mobile devices. The encoder has low computational complexity and the compressed video is robust to transmission errors. The chapter estimates the scheme's encoding complexity and introduces an optimization procedure which minimizes the expected reconstruction distortion subject to the packet loss rate.In Chapter 4, MDC of stereoscopic video is addressed. Two MDC schemes are introduced, one based on spatial scaling and another based on temporal subsampling. A switching criterion makes it possible to decide, which scheme is more advantageous for the sequence being encoded.Chapter 5 discusses MDC of 3D meshes. It introduces two MDC approaches for coding highly detailed 3D meshes. The schemes are able to produce multiple descriptions and are easily adaptable to changing packet loss rate and bit budget. The proposed D-R curve modeling significantly decreases computational load at the preparatory stage.