Towards Machine-interpretable A.I. Policy Representation: An ODRL Approach for A.I. Model Cards

Abstract

Artificial intelligence (AI) and machine learning (ML) systems have been integrated into an increasing number of sensitive sectors of the modern world. Therefore, ensuring transparency in how these systems are developed has become a key requirement, especially following the implementation of regulations such as the EU AI Act. Model Cards, developed by Google, have emerged as a way for developers to document essential information about AI models, including their intended uses, training data, and ethical considerations. However, their current format is often unstructured and lacking in machine-interpretability, which makes it difficult for assessing the risks of AI systems. This thesis aims to address this issue by studying how model cards can be represented using the Open Digital Rights Language (ODRL), a semantic policy expression language. Its two main objectives are to construct an ODRL-based profile for model card representation, and based on this, to convert free-text model cards into ODRL with key information preserved. First, a qualitative and thematic analysis of state-of-the-art model cards was conducted to extract key themes and build the ODRL profile. Subsequently, a proof-of-concept tool was developed to automate the conversion of model cards into their ODRL structure, using a large language model (LLM) approach. The research and implementation show that the ODRL profile effectively captures model cards’ information in a semantic manner, and the prototype can automatically transform model cards into its ODRL representation. The results also reveal several un-solved issues, notably a lack of standardized vocabularies dedicated for model cards’ machine-interpretability, highlighting the need for future works on this matter. The thesis therefore serves as a foundational step towards the long-term objective of enabling AI system risk assessment through machine-interpretable, ODRL-based model cards.