Robust Support Vector Machines For Implicit Outlier Removal

Abstract

The support vector machine is a machine learning algorithm which has been successfully applied to solve classification problems since its introduction in the early 1990s. It is based on the work of Vladimir Vapnik on Statistical Learning Theory and is theoretically well founded. Following the discriminative approach, the SVM yields a classifier which separates two classes by a hyperplane. The training instances are classified according to the sign of their distance to the hyperplane. This hyperplane is defined by a small number of training instances such that the distance of the training instances of both classes to the hyperplane is maximized and the misclassification error is minimized. Hence the support vector machine belongs to the family of maximum margin classifiers. Since the support vector machine does not estimate the underlying class conditional distribution of the training instances, but instead uses them directly to construct the classifier, it is important that the training instances are sampled from the underlying class conditional distribution. If this is not the case because the training set is contaminated with outliers, the accuracy of the classifier defined by the support vector machine decreases. Based on this observation several approaches have been proposed to improve the robustness of the support vector machine against outliers in the training data. In this thesis we will discuss the class robust support vector machines which aim to make the standard support vector machine robust against noise by implicit outlier filtering. Those approaches are using the support vector machine to detect and remove outliers based on their position relative to the separating hyperplane. Since the success of those methods is only empirically proven, we conduct a thoroughly experimental study in order to determine under which conditions those robust methods can be applied in practice. We are especially interested if the additional parameter which controls the removal of outliers can be estimated from a training set which is contamined by outliers.