RoboWCAG: Utilization of Artificial Intelligence in Web Accessibility Test Automation : A constructive research study
Sheitanov, Roni (2025)
2025
Tietojenkäsittelyopin maisteriohjelma - Master's Programme in Computer Science
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
Hyväksymispäivämäärä
2025-11-16
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202510169969
https://urn.fi/URN:NBN:fi:tuni-202510169969
Tiivistelmä
With an estimated 1.3 billion people globally experiencing significant disabilities, the need for accessible digital products and services has increased significantly. The European Accessibility Act and Finland's Digital Services Act mandate compliance with WCAG 2.1 AA standards, creating substantial testing requirements for organizations. Current accessibility testing relies heavily on manual verification, which is time-intensive and resource-demanding, particularly for large software systems. While automated testing libraries exist, they have not yet incorporated artificial intelligence technologies.
This thesis addresses the gap between current automated accessibility testing capabilities and the comprehensive coverage required for WCAG 2.1 compliance. The research investigates how AI-enhanced testing frameworks can improve accessibility validation through three main research questions: examining the practical limitations and opportunities of combining Python-based HTML parsing with AI visual analysis, identifying which WCAG 2.1 success criteria can be tested through various automated approaches, and evaluating the effectiveness of integrating vision-based AI models like Claude Sonnet 3.7 into automated testing workflows.
The methodology involved constructive research through the development of RoboWCAG, a custom library for Robot Framework that converts WCAG 2.1 standards into executable test cases. The library combines traditional HTML parsing and static code analysis with dynamic rendering and AI-assisted visual analysis to create a comprehensive testing framework for web accessibility assessment.
The RoboWCAG library successfully converted 47 out of 78 WCAG 2.1 criteria into automated test keywords, achieving approximately 60% coverage and exceeding existing commercial solutions. Among these implementations, 11 keywords utilized direct API calls to Claude 3.7 Sonnet for complex visual and contextual analysis tasks that traditionally have required manual verification.
RoboWCAG demonstrates a successful translation of WCAG 2.1 criteria into actionable test cases, providing organizations with a more comprehensive tool for accessibility compliance verification. The work contributes to the accessibility testing field by establishing a foundation for AI-integrated testing frameworks and offers practical implications for organizations seeking to meet regulatory accessibility requirements.
This thesis addresses the gap between current automated accessibility testing capabilities and the comprehensive coverage required for WCAG 2.1 compliance. The research investigates how AI-enhanced testing frameworks can improve accessibility validation through three main research questions: examining the practical limitations and opportunities of combining Python-based HTML parsing with AI visual analysis, identifying which WCAG 2.1 success criteria can be tested through various automated approaches, and evaluating the effectiveness of integrating vision-based AI models like Claude Sonnet 3.7 into automated testing workflows.
The methodology involved constructive research through the development of RoboWCAG, a custom library for Robot Framework that converts WCAG 2.1 standards into executable test cases. The library combines traditional HTML parsing and static code analysis with dynamic rendering and AI-assisted visual analysis to create a comprehensive testing framework for web accessibility assessment.
The RoboWCAG library successfully converted 47 out of 78 WCAG 2.1 criteria into automated test keywords, achieving approximately 60% coverage and exceeding existing commercial solutions. Among these implementations, 11 keywords utilized direct API calls to Claude 3.7 Sonnet for complex visual and contextual analysis tasks that traditionally have required manual verification.
RoboWCAG demonstrates a successful translation of WCAG 2.1 criteria into actionable test cases, providing organizations with a more comprehensive tool for accessibility compliance verification. The work contributes to the accessibility testing field by establishing a foundation for AI-integrated testing frameworks and offers practical implications for organizations seeking to meet regulatory accessibility requirements.