Incremental Multi-Domain Learning with Domain-Specific Early Exits
Senhaji, Ali (2020)
2020
Degree Programme in Information Technology, MSc (Tech)
Informaatioteknologian ja viestinnän tiedekunta - Faculty of Information Technology and Communication Sciences
This publication is copyrighted. Only for Your own personal use. Commercial use is prohibited.
Hyväksymispäivämäärä
2020-05-27
Julkaisun pysyvä osoite on
http://urn.fi/URN:NBN:fi:tuni-202004294724
http://urn.fi/URN:NBN:fi:tuni-202004294724
Tiivistelmä
Deep learning architectures can achieve state-of-the-art results in several computer vision tasks. However, these methods are highly specialized, i.e., for every task from a new domain, an independent new model is required. Multi-domain learning investigates new ways of developing one model capable of solving different tasks from different domains. Most of the multi-domain methods try to maximize the parameter sharing, i.e., domain agnostic parameters or base model, and minimize domain-specific parameters. In this archetype, all of the target domains utilize all of the base network. Given that domains come in different levels of difficulty, this leads to inefficient use of the base model to solve tasks in easier domains. In this thesis, we examine the adaptive use of the base model parameters.
We propose a novel adaptive approach for incremental multi-domain learning, where different parts of the base network are adapted depending on the level of complexity of each individual domain. The aim is to reach an efficient use of the base network while maintaining a high performance. Developing efficient models with the most optimal capacity is important for a multitude of applications. The proposed adaptive method achieves comparable performance to adapting the whole base network for easier domains while reducing by far the number of parameters. This leads to efficient multi-domain learning solutions and can be useful in many applications (e.g., budget inference within edge devices).
We investigated the use of the proposed approach for residual networks. High performance comparable to using the whole network was achieved for domains with easy and intermediate levels of difficulty, with only 4.7% and 23.8% of the parameters, respectively. We have used a benchmark of ten visually different datasets, to solve problems including recognizing handwritten characters, classifying flowers, detecting pedestrians, classifying aircraft, and detecting actions from real-life video snapshots. Our adaptive method achieved a mean accuracy of 72.79%, using only 15% of the parameters required to have ten different fine-tuned networks, compared to 73.44% mean accuracy. Thus, our results confirm our hypothesis that it is not necessary to use the whole base network for all the domains, and easier domains can be more efficiently parameterized with the proposed method.
We propose a novel adaptive approach for incremental multi-domain learning, where different parts of the base network are adapted depending on the level of complexity of each individual domain. The aim is to reach an efficient use of the base network while maintaining a high performance. Developing efficient models with the most optimal capacity is important for a multitude of applications. The proposed adaptive method achieves comparable performance to adapting the whole base network for easier domains while reducing by far the number of parameters. This leads to efficient multi-domain learning solutions and can be useful in many applications (e.g., budget inference within edge devices).
We investigated the use of the proposed approach for residual networks. High performance comparable to using the whole network was achieved for domains with easy and intermediate levels of difficulty, with only 4.7% and 23.8% of the parameters, respectively. We have used a benchmark of ten visually different datasets, to solve problems including recognizing handwritten characters, classifying flowers, detecting pedestrians, classifying aircraft, and detecting actions from real-life video snapshots. Our adaptive method achieved a mean accuracy of 72.79%, using only 15% of the parameters required to have ten different fine-tuned networks, compared to 73.44% mean accuracy. Thus, our results confirm our hypothesis that it is not necessary to use the whole base network for all the domains, and easier domains can be more efficiently parameterized with the proposed method.