Structured Deep Learning for Fine-grained Visual Classification

Abstract

We propose a structured decision making approach using privileged information that improves the popular deep convolutional neural network (DCNN) methodology for visual class detection. This is achieved by discovering and exploiting additional - privileged - information available only during training. We instantiate learning with privileged information by defining “latent sub-tasks” that indirectly contribute to the main task – fine-grained visual classification. Specifically, detection of the object location, detection and selection of the object parts or detection of a semantic super-class are examples of la- tent subtasks which we exploit. In the experiments, our framework using deep privileged parts consistently improves the performance of fine-grained classification and our results are comparable to or better than the state-of-the-art methods without requiring expensive human efforts to provide additional annotations on object parts in both training and testing phases, which is thus suitable for scaling to large-scale data owing to its part-annotation-free manner.