Projects per year
Abstract
Generalising robustly to distribution shift is a major challenge that is pervasive across most realworld applications of machine learning. A recent study highlighted that many advanced algorithms proposed to tackle such domain generalisation (DG) fail to outperform a properly tuned empirical risk minimisation (ERM) baseline. We take a different approach, and explore the impact of the ERM loss function on out-of-domain generalisation. In particular, we introduce a novel metalearning approach to loss function search based on implicit gradient. This enables us to discover a general purpose parametric loss function that provides a drop-in replacement for cross-entropy. Our loss can be used in standard training pipelines to efficiently train robust models using any neural architecture on new datasets. The results show that it clearly surpasses cross-entropy, enables simple ERM to outperform some more complicated prior DG methods, and provides excellent performance across a variety of DG benchmarks. Furthermore, unlike most existing DG approaches, our setup applies to the most practical setting of single-source domain generalisation, on which we show significant improvement.
Original language | English |
---|---|
Title of host publication | Proceedings of the 39th International Conference on Machine Learning |
Publisher | PMLR |
Pages | 7002-7016 |
Number of pages | 15 |
Volume | 162 |
Publication status | Published - 23 Jul 2022 |
Event | 39th International Conference on Machine Learning - Baltimore, United States Duration: 17 Jul 2022 → 23 Jul 2022 Conference number: 39 https://icml.cc/Conferences/2022 |
Publication series
Name | Proceedings of Machine Learning Research |
---|---|
Publisher | MIT Press |
Volume | 162 |
ISSN (Electronic) | 2640-3498 |
Conference
Conference | 39th International Conference on Machine Learning |
---|---|
Abbreviated title | ICML 2022 |
Country/Territory | United States |
City | Baltimore |
Period | 17/07/22 → 23/07/22 |
Internet address |
Fingerprint
Dive into the research topics of 'Loss Function Learning for Domain Generalization by Implicit Gradient'. Together they form a unique fingerprint.Projects
- 1 Finished
-
Signal Processing in the Information Age
Davies, M. (Principal Investigator), Hopgood, J. (Co-investigator), Hospedales, T. (Co-investigator), Mulgrew, B. (Co-investigator), Thompson, J. (Co-investigator), Tsaftaris, S. (Co-investigator) & Yaghoobi Vaighan, M. (Co-investigator)
1/07/18 → 31/03/24
Project: Research