CALMED - Contrastive Alignment of Language and (Symbolic) Music EDits : Text-Driven Symbolic Music Editing
Villagran, Daniel (2026)
2026
Master's Programme in Computing Sciences and Electrical Engineering
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
Hyväksymispäivämäärä
2026-06-25
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-202606247939
https://urn.fi/URN:NBN:fi:tuni-202606247939
Tiivistelmä
We designed and trained CALMED, a model that generates modified MIDI files from an input MIDI file and a natural-language editing instruction. CALMED utilizes a general formulation that can theoretically learn to implement any arbitrary transformation, if provided with the right dataset. However, given that this is a novel approach and that the difficulty of dataset generation scales with target transformation complexity, we constrain this project to the domain of simple probabilistic pitch shift transformations as an initial testbed.
Our model uses a novel Cross-Modal Delta Contrastive Learning approach to transpose notes in MIDI files according to the user’s textual directions. We use contrastive learning to learn the alignment between text embeddings and delta MIDI embeddings. Delta MIDI embeddings represent the difference between a source MIDI embedding and its transformed counterpart, allowing the model to learn relationships between textual instructions and musical edits.
On a held-out test set, CALMED achieved an average probability similarity score of 0.85 compared with 0.69 for a random baseline, while preserving note count and rhythmic structure with low error. However, improvements in shift set recovery were limited, indicating that the model captures transformation likelihood more reliably than the specific pitch shift operations requested. The results demonstrate the feasibility of learning text-conditioned MIDI transformations using cross-modal delta representations. Future work should investigate alternative encoder-decoder architectures, sampling strategies, and loss formulations.
All code and model checkpoints from this work are publicly available at https://github.com/DVILL17/Thesis-2025-2026.
Our model uses a novel Cross-Modal Delta Contrastive Learning approach to transpose notes in MIDI files according to the user’s textual directions. We use contrastive learning to learn the alignment between text embeddings and delta MIDI embeddings. Delta MIDI embeddings represent the difference between a source MIDI embedding and its transformed counterpart, allowing the model to learn relationships between textual instructions and musical edits.
On a held-out test set, CALMED achieved an average probability similarity score of 0.85 compared with 0.69 for a random baseline, while preserving note count and rhythmic structure with low error. However, improvements in shift set recovery were limited, indicating that the model captures transformation likelihood more reliably than the specific pitch shift operations requested. The results demonstrate the feasibility of learning text-conditioned MIDI transformations using cross-modal delta representations. Future work should investigate alternative encoder-decoder architectures, sampling strategies, and loss formulations.
All code and model checkpoints from this work are publicly available at https://github.com/DVILL17/Thesis-2025-2026.