Generating Domain-Specific Language Code with a Fine-Tuned Language Model : A Case Study on Terraform Infrastructure as Code
Bayoud, Walid (2026)
2026
Master's Programme in Computing Sciences and Electrical Engineering
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
Hyväksymispäivämäärä
2026-07-31
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202607318667
https://urn.fi/URN:NBN:fi:tuni-202607318667
Tiivistelmä
Large language models have substantially advanced automated code generation, yet their accuracy degrades sharply on domain-specific languages for which little public training data exists. Infrastructure-as-Code languages are a prominent example: even state-of-the-art models generate valid configurations only a fraction of the time. This thesis studies whether a small, openly available code model can be adapted to one such domain-specific language — the HashiCorp Configuration Language used by Terraform — through parameter-efficient fine-tuning.
A pipeline is designed in which the Qwen2.5-Coder model is fine-tuned with QLoRA, a quantised low-rank adaptation method that trains only a small set of additional parameters while the pretrained weights remain frozen, allowing the entire procedure to run on a single graphics processing unit. Because publicly available Infrastructure-as-Code configurations are scarce, a dataset of natural-language-to-configuration pairs is constructed from Terraform provider documentation. The correctness of the model output is then assessed automatically using Terraform’s own validation tooling as a programmatic oracle.
The work follows an Action Research methodology, in which the dataset, the fine-tuning configuration and the evaluation procedure are refined over successive iterations. The thesis contributes a reproducible data-construction procedure, a complete fine-tuning configuration for adapting an open-weight code model to the HashiCorp Configuration Language, and an automated validation-based evaluation together with a discussion of its limitations. Although Terraform serves as the case study, the methodology is intended to generalise to other low resource technical domains in which data scarcity and weak evaluation signals are the principal obstacles to applying language models effectively.
A pipeline is designed in which the Qwen2.5-Coder model is fine-tuned with QLoRA, a quantised low-rank adaptation method that trains only a small set of additional parameters while the pretrained weights remain frozen, allowing the entire procedure to run on a single graphics processing unit. Because publicly available Infrastructure-as-Code configurations are scarce, a dataset of natural-language-to-configuration pairs is constructed from Terraform provider documentation. The correctness of the model output is then assessed automatically using Terraform’s own validation tooling as a programmatic oracle.
The work follows an Action Research methodology, in which the dataset, the fine-tuning configuration and the evaluation procedure are refined over successive iterations. The thesis contributes a reproducible data-construction procedure, a complete fine-tuning configuration for adapting an open-weight code model to the HashiCorp Configuration Language, and an automated validation-based evaluation together with a discussion of its limitations. Although Terraform serves as the case study, the methodology is intended to generalise to other low resource technical domains in which data scarcity and weak evaluation signals are the principal obstacles to applying language models effectively.