A denoising convolutional neural network for self-supervised rank effectiveness estimation on image retrieval

• Main Author: Valem, Lucas Pascotti [UNESP]
• Publication Date: 2021
• Other Authors: Pedronette, Daniel Carlos Guimarães [UNESP]
• Format: Conference object
• Language: eng
• Source: Repositório Institucional da UNESP
• Download full: http://dx.doi.org/10.1145/3460426.3463645; http://hdl.handle.net/11449/222408
• topic: Content-based image retrieval; Convolutional neural networks; Denoising; Effectiveness estimation; Query performance prediction; Self-supervised learning; Unsupervised learning

Image and multimedia retrieval has established as a prominent task in an increasingly digital and visual world. Mainly supported by decades of development on hand-crafted features and the success of deep learning techniques, various different feature extraction and retrieval approaches are currently available. However, the frequent requirements for large training sets still remain as a fundamental bottleneck, especially in real-world and large-scale scenarios. In the scarcity or absence of labeled data, choosing what retrieval approach to use became a central challenge. A promising strategy consists in to estimate the effectiveness of ranked lists without requiring any groundtruth data. Most of the existing measures exploit statistical analysis of the ranked lists and measure the reciprocity among lists of images in the top positions. This work innovates by proposing a new and self-supervised method for this task, the Deep Rank Noise Estimator (DRNE). An algorithm is presented for generating synthetic ranked list data, which is modeled as images and provided for training a Convolutional Neural Network that we propose for effectiveness estimation. The proposed model is a variant of the DnCNN (Denoiser CNN), which intends to interpret the incorrectness of a ranked list as noise, which is learned by the network. Our approach was evaluated on 5 public image datasets and different tasks, including general image retrieval and person re-ID. We also exploited and evaluated the complementary between the proposed approach and related rank-based approaches through fusion strategies. The experimental results showed that the proposed method is capable of achieving up to 0.88 of Pearson correlation with MAP measure in general retrieval scenarios and 0.74 in person re-ID scenarios.