Light Field Compression Using Disparity-Based 4D Predictive Coding

Abstract

The contributions of this thesis are new modeling and compression algorithms for stereo images, disparity maps, and light field images. Plenoptic imaging is an emerging area of digital imaging with growing interest in both research and applications. In particular, research in light field coding is very active, since traditional image coding solutions are not designed to fully exploit the multi-dimensional characteristics of light field images. We show that excellent coding performance can be achieved in the area of light field coding through new architectures combining: disparity-based view warping, sparse linear prediction, and entropy coding of prediction errors.

Departing from traditional digital images having two-dimensional discrete spatial sampling, in this thesis we consider light fields represented by four-dimensional discretely sampled plenoptic function, which allows the sampling and rendering of the scene from several angular directions. The corresponding angular views are highly redundant, depicting the scene from slightly different view points or camera positions. Any given angular view can be reconstructed from a single or a set of reference angular views, using accurate estimation of disparity maps. The methods of this thesis are exploiting the four-dimensional redundancy of the light field for producing very efficient coding.

We first present new methods for lossless coding of stereo images and lossy coding of light fields using sparse linear predictive models and context adaptive arithmetic coding. We also present a novel algorithm for lossless coding of high resolution disparity maps, and propose a maximum likelihood estimator for normalized disparity maps, which models pairwise disparity maps and relative positions for all angular views in a light field. We present two novel algorithms for light field coding using hierarchical light field reconstruction based on normalized disparity maps and sparse linear predictive models, and demonstrate state-of-the-art performance in lossy light field coding. Further, the presented light field coding algorithms have been successfully proposed for international standardization leading to the creation of the 4D prediction profile of the forthcoming ISO/IEC 21794 standard for light field coding.