Projects per year
Abstract
Dataset condensation aims at reducing the network training effort through condensing a cumbersome training set into a compact synthetic one. State-of-the-art approaches largely rely on learning the synthetic data by matching the gradients between the real and synthetic data batches. Despite the intuitive motivation and promising results, such gradient-based methods, by nature, easily overfit to a biased set of samples that produce dominant gradients, and thus lack global supervision of data distribution. In this paper, we propose a novel scheme to Condense dataset by Aligning FEatures (CAFE), which explicitly attempts to preserve the real-feature distribution as well as the discriminant power of the resulting synthetic set, lending itself to strong generalization capability to various architectures. At the heart of our approach is an effective strategy to align features from the real and synthetic data across various scales, while accounting for the classification of real samples. Our scheme is further backed up by a novel dynamic bi-level optimization, which adaptively adjusts parameter updates to prevent over-/under-fitting. We validate the proposed CAFE across various datasets, and demonstrate that it generally outperforms the state of the art: on the SVHN dataset, for example, the performance gain is up to 11%. Extensive experiments and analyses verify the effectiveness and necessity of proposed designs.
Original language | English |
---|---|
Title of host publication | Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
Publisher | Institute of Electrical and Electronics Engineers |
Pages | 12186-12195 |
Number of pages | 10 |
ISBN (Electronic) | 978-1-6654-6946-3 |
ISBN (Print) | 978-1-6654-6947-0 |
DOIs | |
Publication status | Published - 27 Sept 2022 |
Event | IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 - New Orleans, United States Duration: 19 Jun 2022 → 24 Jun 2022 https://cvpr2022.thecvf.com/ |
Publication series
Name | IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) |
---|---|
Publisher | IEEE |
ISSN (Print) | 1063-6919 |
ISSN (Electronic) | 2575-7075 |
Conference
Conference | IEEE/CVF Conference on Computer Vision and Pattern Recognition 2022 |
---|---|
Abbreviated title | CVPR 2022 |
Country/Territory | United States |
City | New Orleans |
Period | 19/06/22 → 24/06/22 |
Internet address |
Keywords / Materials (for Non-textual outputs)
- Computer Vision
- Machine Learning (ML)
- dataset synthesis
- dataset condensation
Fingerprint
Dive into the research topics of 'CAFE Learning to Condense Dataset by Aligning Features'. Together they form a unique fingerprint.Projects
- 1 Active
-
Visual AI: An Open World Interpretable Visual Transformer
Bilen, H. (Principal Investigator)
Engineering and Physical Sciences Research Council
1/12/20 → 30/11/26
Project: Research