Projects per year
Generalising robustly to distribution shift is a major challenge that is pervasive across most realworld applications of machine learning. A recent study highlighted that many advanced algorithms proposed to tackle such domain generalisation (DG) fail to outperform a properly tuned empirical risk minimisation (ERM) baseline. We take a different approach, and explore the impact of the ERM loss function on out-of-domain generalisation. In particular, we introduce a novel metalearning approach to loss function search based on implicit gradient. This enables us to discover a general purpose parametric loss function that provides a drop-in replacement for cross-entropy. Our loss can be used in standard training pipelines to efficiently train robust models using any neural architecture on new datasets. The results show that it clearly surpasses cross-entropy, enables simple ERM to outperform some more complicated prior DG methods, and provides excellent performance across a variety of DG benchmarks. Furthermore, unlike most existing DG approaches, our setup applies to the most practical setting of single-source domain generalisation, on which we show significant improvement.
|Title of host publication||Proceedings of the 39th International Conference on Machine Learning|
|Number of pages||15|
|Publication status||Accepted/In press - 30 May 2022|
|Event||39th International Conference on Machine Learning - Baltimore, United States|
Duration: 17 Jul 2022 → 23 Jul 2022
Conference number: 39
|Conference||39th International Conference on Machine Learning|
|Abbreviated title||ICML 2022|
|Period||17/07/22 → 23/07/22|
FingerprintDive into the research topics of 'Loss Function Learning for Domain Generalization by Implicit Gradient'. Together they form a unique fingerprint.
- 1 Active
Signal Processing in the Information Age
Davies, M., Hopgood, J., Hospedales, T., Mulgrew, B., Thompson, J., Tsaftaris, S. & Yaghoobi Vaighan, M.
1/07/18 → 31/03/24