Multi-label Text Classification with Deep Learning Models

Abstract

Multi-label text classification (ML TC) assigns multiple, potentially dependent labels to each document and underpins applications such as medical diagnosis, legal tag­ging, and news categorization. In general, ML TC must handle correlated and often hierarchical labels, severe class imbalance, and long-tailed distributions. Two main approaches exist: binary relevance (BR) and multi-label (ML) methods. While BR methods are simple and scalable, they ignore label dependencies. Conversely, ML methods lack general guidelines and can sometimes underperform BR in practice.

This thesis evaluates BR for multi-label text classification in modern deep learning. Across diverse datasets, a convolutional BR model (BR-CNN) outperforms traditional BR methods. We further test whether CNNs that model label depen­dencies can close this gap. Two variants-Implicit-Threshold CNN (IT-CNN) and Adaptive-Threshold CNN (AT-CNN) -are compared with a standard multi-label CNN (M-CNN) and BR-CNN. Results show that dependency-focused models are not always guaranteed to exceed BR-CNN performance on multi-label datasets.

Another contribution of this thesis is the Partial-Dependency Binary Relevance (PDBR-CNN) method, a binary relevance architecture that selectively encodes par­tial label dependencies while retaining the robustness and scalability of BR. Across benchmark datasets, PDBR-CNN achieves substantial improvements over classical BR variants (e.g., classifier chains, stacking) and delivers competitive, often superior, performance compared to recent deep multi-label models.

Overall, this thesis systematically examines the performance of deep learning based BR methods and introduces novel CNN-based architectures. The results show that BR can be extended to incorporate deep neural modeling and label-dependency information, achieving superior performance with reasonable runtime.