Fairness- and Explainability-Aware Streaming Entity Resolution

Abstract

Entity Resolution (ER) is a fundamental task in data integration, aiming to identify records that refer to the same real-world entity across heterogeneous data sources. While significant advances have been achieved in both rule-based and machine learning-based ER, the increasing prevalence of streaming data introduces new challenges. In streaming environments, ER systems must operate under continuous data arrival, bounded memory, and real-time constraints. At the same time, growing societal and regulatory demands require ER pipelines to be not only efficient, but also fair and explainable. However, existing research largely addresses scalability, fairness, and explainability in isolation, resulting in fragmented solutions that fail to provide an integrated and responsible ER workflow.

This thesis investigates how ER can be redesigned to operate efficiently over streaming data while incorporating fairness and explainability as first-class design objectives. First, it introduces an incremental blocking framework that enables scalable candidate generation under streaming constraints. By combining attribute selection, similarity-graph construction, and top-<i>n</i> neighborhood control, the proposed approach preserves effectiveness while significantly improving efficiency and robustness in dynamic environments.

Building upon this computational backbone, the thesis proposes a fairness-aware streaming ER workflow that embeds fairness constraints directly into the matching and ranking process. Rather than applying fairness as a post-hoc correction, the proposed workflow integrates proportional and bias-aware mechanisms into the resolution pipeline, demonstrating that fairness improvements can be achieved without substantial loss in accuracy or scalability.

To further enhance transparency, the thesis introduces an explanation-driven extension that integrates explanation signals into the resolution process itself. By leveraging explanation scores to moderate ranking decisions, the proposed workflow strengthens interpretability while maintaining both fairness and performance, particularly in heterogeneous and noisy domains. Experimental results show that the proposed workflow improved interpretability, increasing explanation by up to 20%, while reducing group-level disparities, on average, by 60% and preserving high matching precision across all evaluated datasets.

Finally, the thesis provides a systematic analysis of existing ER methods under the joint perspective of efficiency, fairness, and explainability. This conceptual synthesis clarifies existing research gaps and establishes a unified framework for responsible streaming ER. Overall, this work demonstrates that efficiency, fairness, and explainability can be jointly operationalized in streaming ER systems.By advancing both the algorithmic and conceptual foundations of ER, this thesis positions Entity Resolution as a transparent, scalable, and ethically grounded component of modern data-driven infrastructures.