On Statistical Modelling and Hypothesis Testing by Information Theoretic Methods

Abstract

The main objective of this thesis is to study various information theoretic methods and criteria in the context of statistical model selection. The focus in this research is on Rissanen’s Minimum Description Length (MDL) principle and its variants, with a special emphasis on the Normalized Maximum Likelihood (NML).<br/><br/>We extend the Rissanen methodology for coping with infinite parametric complexity and discuss two particular cases. This is applied for deriving four NMLcriteria and investigate their performance. Furthermore, we find the connection between Stochastic Complexity (SC), defined as minus logarithm of NML, and other model selection criteria.<br/><br/>We also study the use of information theoretic criteria (ITC) for selecting the order of autoregressive (AR) models in the presence of nonstationarity. In particular, we give a modified version of Sequentially NML (SNML) when the model parameters are estimated by forgetting factor LS algorithm.<br/><br/>Another contribution of the thesis is in connection with the new approach for composite hypothesis testing using Optimally Distinguishable Distributions (ODD). The ODD-detector for subspace signals in Gaussian noise is introduced and its performance is evaluated.<br/><br/>Additionally, we exploit the Kolmogorov Structure Function (KSF) to derive a new criterion for cepstral nulling, which has been recently applied to the problem of periodogram smoothing.<br/><br/>Finally, the problem of fairness in multiaccess communication systems is investigated and a new method is proposed. The new approach is based on partitioning the network into subnetworks and employing two different multiple-access schemes within and across subnetworks. It is also introduced an algorithm for selecting optimally the subnetworks such that to achieve the max-min fairness.