Generation and Generalized Detection of Fully Synthetic Photorealistic Images
Olán, Toni (2023)
2023
Tietotekniikan DI-ohjelma - Master's Programme in Information Technology
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2023-11-22
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:tuni-2023112410231
https://urn.fi/URN:NBN:fi:tuni-2023112410231
Tiivistelmä
Image generation has reached a point where a machine learning model can create an image that looks like it could have been captured with a camera. People using these models can limitedly control the creation process, quickly synthesizing images from text. Although the controlled synthesis of images opens up a great number of creative possibilities, it can also be used for nefarious purposes, such as identity theft. Being able to detect if an image is synthetic could help in mitigating possible harm. This thesis studies the methods used for both creating and detecting synthetic images by performing a literature review for both topics. The main goals are to find the key ideas, recent advancements, and challenges behind synthetic image generators and detectors, and test how generalizable current open-source pretrained detectors are.
Generative adversarial networks, autoregressive models, and diffusion models were recognized as the most popular generation methods. Although their implementations differ, they face similar challenges, such as the need for large datasets, which produce biases and ethical concerns regarding the use of the generators. For detection, multiple methods have been suggested, with many distinguishing patterns both in synthetic images and those captured with digital cameras. Watermarking was also identified as a detection solution, but it is difficult to implement universally.
Experiments were conducted on open-source detectors, where synthetic and real images went through image editing processes similar to those in social media. The implementation of the experiments highlighted issues for the generation and detection processes, such as the limited availability of text-to-image generators and detectors. The results showed that at least the studied detectors are not currently very reliable, even when trying to detect modified images synthesized by the same generative methods whose images the detectors were trained to classify.
Generative adversarial networks, autoregressive models, and diffusion models were recognized as the most popular generation methods. Although their implementations differ, they face similar challenges, such as the need for large datasets, which produce biases and ethical concerns regarding the use of the generators. For detection, multiple methods have been suggested, with many distinguishing patterns both in synthetic images and those captured with digital cameras. Watermarking was also identified as a detection solution, but it is difficult to implement universally.
Experiments were conducted on open-source detectors, where synthetic and real images went through image editing processes similar to those in social media. The implementation of the experiments highlighted issues for the generation and detection processes, such as the limited availability of text-to-image generators and detectors. The results showed that at least the studied detectors are not currently very reliable, even when trying to detect modified images synthesized by the same generative methods whose images the detectors were trained to classify.