Visual Knowledge Tracing

Neehar Kondapaneni, Pietro Perona, Oisin Mac Aodha

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Each year, thousands of people learn new visual categorization tasks -- radiologists learn to recognize tumors, birdwatchers learn to distinguish similar species, and crowd workers learn how to annotate valuable data for applications like autonomous driving. As humans learn, their brain updates the visual features it extracts and attend to, which ultimately informs their final classification decisions. In this work, we propose a novel task of tracing the evolving classification behavior of human learners as they engage in challenging visual classification tasks. We propose models that jointly extract the visual features used by learners as well as predicting the classification functions they utilize. We collect three challenging new datasets from real human learners in order to evaluate the performance of different visual knowledge tracing methods. Our results show that our recurrent models are able to predict the classification behavior of human learners on three challenging medical image and species identification tasks.
Original languageEnglish
Title of host publicationComputer Vision – ECCV 2022: 17th European Conference, Tel Aviv, Israel, October 23–27, 2022, Proceedings, Part XXV
EditorsShai Avidan, Gabriel Brostow, Moustapha Cissé, Giovanni Maria Farinella, Tal Hassner
PublisherSpringer
Pages415-431
Number of pages17
ISBN (Electronic)978-3-031-19806-9
ISBN (Print)978-3-031-19805-2
DOIs
Publication statusPublished - 20 Oct 2022
EventEuropean Conference on Computer Vision 2022 - Israel, Tel Aviv, Israel
Duration: 23 Oct 202227 Oct 2022
https://eccv2022.ecva.net/

Publication series

NameLecture Notes in Computer Science
PublisherSpringer Cham
Volume13685
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Computer Vision 2022
Abbreviated titleECCV 2022
Country/TerritoryIsrael
CityTel Aviv
Period23/10/2227/10/22
Internet address

Keywords / Materials (for Non-textual outputs)

  • visual classification
  • knowledge tracing
  • human learning

Fingerprint

Dive into the research topics of 'Visual Knowledge Tracing'. Together they form a unique fingerprint.

Cite this