Predicting image social tags using a convolutional neural network

Abstract

Convolutional Neural Networks (CNN) are regarded as the state-of-the-art approach in image classification tasks, i.e. predicting appropriate labels for an image. Active ongoing research in the field has produced methods that rival human accuracy in image recognition. At the same time, the popularity of social networking sites has led to a huge number of user-uploaded photographs labeled with social tags or #hashtags.

This thesis provides an introduction to deep artificial neural networks, specifically when employed for the task of image recognition. The learnings are then applied to the development of a CNN trained on images and social tags collected from social networking sites. The goal is to predict relevant social tags for a newly uploaded photograph.

An evaluation of the proposed implemented approach reveals that it is indeed feasible to use a CNN to predict relevant tags for an image, but reaching high accuracy metrics is difficult. Further research is needed to improve the quality of the predictions.