Projects per year
Abstract / Description of output
Scene graph generation (SGG) aims to capture a wide variety of interactions between pairs of objects, which is essential for full scene understanding. Existing SGG methods trained on the entire set of relations fail to acquire complex reasoning about visual and textual correlations due to various biases in training data. Learning on trivial relations that indicate generic spatial configuration like 'on' instead of informative relations such as 'parked on' does not enforce this complex reasoning, harming generalization. To address this problem, we propose a novel framework for SGG training that exploits relation labels based on their informativeness. Our model-agnostic training procedure imputes missing informative relations for less informative samples in the training data and trains a SGG model on the imputed labels along with existing annotations. We show that this approach can successfully be used in conjunction with state-of-the-art SGG methods and improves their performance significantly in multiple metrics on the standard Visual Genome benchmark. Furthermore, we obtain considerable improvements for unseen triplets in a more challenging zero-shot setting.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
Publisher | Institute of Electrical and Electronics Engineers |
Pages | 15575-15585 |
Number of pages | 11 |
ISBN (Electronic) | 978-1-6654-6946-3 |
ISBN (Print) | 978-1-6654-6947-0 |
DOIs | |
Publication status | Published - 27 Sept 2022 |
Event | IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 - New Orleans, United States Duration: 19 Jun 2022 → 24 Jun 2022 https://cvpr2022.thecvf.com/ |
Publication series
Name | IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
---|---|
Publisher | IEEE |
ISSN (Print) | 1063-6919 |
ISSN (Electronic) | 2575-7075 |
Conference
Conference | IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 |
---|---|
Abbreviated title | CVPR 2022 |
Country/Territory | United States |
City | New Orleans |
Period | 19/06/22 → 24/06/22 |
Internet address |
Keywords / Materials (for Non-textual outputs)
- scene graph generation
- semi-supervised learning
Fingerprint
Dive into the research topics of 'Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation'. Together they form a unique fingerprint.Projects
- 1 Active
-
Visual AI: An Open World Interpretable Visual Transformer
Engineering and Physical Sciences Research Council
1/12/20 → 30/11/26
Project: Research