MARV: A Multi-Agent Approach for Validation of Software Requirements Specifications

Abstract

This thesis presents an experimental framework (MARV) using LLMs and structured role-based agent workflows with retrieval-augmented generation (RAG) that empowers LLMs to dynamically access and utilize a knowledge base of relevant documents, enabling them to provide contextually rich and informative responses to from IEEE/ISO and industry-specific standards for validating natural language software requirements more effectively. The background behind this work is the persistent challenge in software engineering, like writing requirements that are clear, complete, and testable. Traditional validation methods, such as manual reviews and rule-based tools with standards and checklists, are time-consuming, subjective, and often overlooked in practice. To address this, MARV automates requirement validation while simulating the industry experts using multi-agents, similar to how different team members would evaluate each requirement. The framework introduces three specialized agents, like a developer agent, a software architect agent, and a product manager agent, each representing an important role in software teams. These agents assess the same requirement sequentially, offering feedback from their respective perspectives. Their reasoning is coordinated using LangGraph, and each agent's validation process is supported by context retrieved from ISO/IEC/IEEE 29148:2018 and organizational standards using a vector-based search system and semantic embeddings generated by sentence transformers.

The system was implemented locally and evaluated using a dataset of 113 real-world software requirements. Its performance was compared against OpenReq PRS, a baseline tool for structural analysis. MARV demonstrated improved issue detection, more meaningful justifications, and stronger alignment with formal standards. While MARV achieved higher recall, successfully identifying a broader range of problematic requirements, it is sensitive to precision, occasionally flagging acceptable requirements as issues because of checking with multiple perspectives. These results suggest that although the system provides richer and more standards-aware feedback, further tuning is needed to improve selectivity and reduce over-flagging. Overall, the findings highlight the potential of intelligent, role-driven validation systems to support early-stage requirements engineering with greater depth, consistency, and traceability.