Using Fairness Metrics as Decision-Making Procedures: Algorithmic Fairness and the Problem of Action-Guidance

Abstract

Frameworks for fair machine learning are envisioned to play an important practical role in the evaluation, training, and selection of machine learning models. In particular, fairness metrics are meant to provide responsible agents with actionable standards for evaluating ML models and conditions which those models should achieve. However, recent studies suggest that fair ML frameworks and metrics do not provide sufficient and actionable guidance for agents. This short paper outlines the main content of a working paper wherein I draw lessons from philosophical debates concerning action-guidance to build a conceptual account that can be applied to analyze whether and when fair ML frameworks and metrics can generate determinate evaluations of fairness and actionable prescriptions for model selection.