Abstract / Description of output
Zero-Shot action recognition is the task of recognizing action classes without visual examples. The problem can be seen as learning a representation on seen classes which generalizes well to instances of unseen classes, without losing discriminability between classes. Neural networks are able to model highly complex boundaries between visual classes, which explains their success as supervised models. However, in Zero-Shot learning, these highly specialized class boundaries may overfit to the seen classes and not transfer well from seen to unseen classes. We propose a novel cluster-based representation, which regularizes the learning process, yielding a representation that generalizes well to instances from unseen classes. We optimize the clustering using reinforcement learning, which we observe is critical. We call the proposed method CLASTER and observe that it consistently outperforms the state-of-the-art in all standard Zero-Shot video datasets, including UCF101, HMDB51 and Olympic Sports; both in the standard Zero-Shot evaluation and the generalized Zero-Shot learning. We see improvements of up to 11.9% over SOTA.
Project Page: https://sites.google.com/view/claster-zsl/home
Project Page: https://sites.google.com/view/claster-zsl/home
Original language | English |
---|---|
Title of host publication | Computer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XX |
Editors | Shai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner |
Publisher | Springer |
Pages | 187-203 |
Number of pages | 17 |
ISBN (Electronic) | 978-3-031-20044-1 |
ISBN (Print) | 978-3-031-20043-4 |
DOIs | |
Publication status | Published - 20 Oct 2022 |
Event | European Conference on Computer Vision 2022 - Israel, Tel Aviv, Israel Duration: 23 Oct 2022 → 27 Oct 2022 https://eccv2022.ecva.net/ |
Publication series
Name | Lecture Notes in Computer Science |
---|---|
Publisher | Springer Cham |
Volume | 13680 |
ISSN (Print) | 0302-9743 |
ISSN (Electronic) | 1611-3349 |
Conference
Conference | European Conference on Computer Vision 2022 |
---|---|
Abbreviated title | ECCV 2022 |
Country/Territory | Israel |
City | Tel Aviv |
Period | 23/10/22 → 27/10/22 |
Internet address |
Keywords / Materials (for Non-textual outputs)
- Zero-Shot
- Clustering
- Action Recognition