Projects per year
Abstract
Generalising robustly to distribution shift is a major challenge that is pervasive across most realworld applications of machine learning. A recent study highlighted that many advanced algorithms proposed to tackle such domain generalisation (DG) fail to outperform a properly tuned empirical risk minimisation (ERM) baseline. We take a different approach, and explore the impact of the ERM loss function on out-of-domain generalisation. In particular, we introduce a novel metalearning approach to loss function search based on implicit gradient. This enables us to discover a general purpose parametric loss function that provides a drop-in replacement for cross-entropy. Our loss can be used in standard training pipelines to efficiently train robust models using any neural architecture on new datasets. The results show that it clearly surpasses cross-entropy, enables simple ERM to outperform some more complicated prior DG methods, and provides excellent performance across a variety of DG benchmarks. Furthermore, unlike most existing DG approaches, our setup applies to the most practical setting of single-source domain generalisation, on which we show significant improvement.
Original language | English |
---|---|
Title of host publication | Proceedings of the 39th International Conference on Machine Learning |
Number of pages | 15 |
Publication status | Accepted/In press - 30 May 2022 |
Event | 39th International Conference on Machine Learning - Baltimore, United States Duration: 17 Jul 2022 → 23 Jul 2022 Conference number: 39 https://icml.cc/ |
Conference
Conference | 39th International Conference on Machine Learning |
---|---|
Abbreviated title | ICML 2022 |
Country/Territory | United States |
City | Baltimore |
Period | 17/07/22 → 23/07/22 |
Internet address |
Fingerprint
Dive into the research topics of 'Loss Function Learning for Domain Generalization by Implicit Gradient'. Together they form a unique fingerprint.Projects
- 1 Active
-
Signal Processing in the Information Age
Davies, M., Hopgood, J., Hospedales, T., Mulgrew, B., Thompson, J., Tsaftaris, S. & Yaghoobi Vaighan, M.
1/07/18 → 31/03/24
Project: Research