An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data

Spandana Gella; Frank Keller

doi:10.18653/v1/N18-2119

An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

Recent research in language and vision has developed models for predicting and disambiguating verbs from images. Here, we ask whether the predictions made by such models correspond to human intuitions about visual verbs. We show that the image regions a verb prediction model identifies as salient for a given verb correlate with the regions fixated by human observers performing a verb classification task.

Original language	English
Title of host publication	The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Place of Publication	New Orleans, Louisiana
Publisher	Association for Computational Linguistics
Pages	758-763
Number of pages	6
DOIs	https://doi.org/10.18653/v1/N18-2119
Publication status	Published - 30 Jun 2018
Event	16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies - Hyatt Regency New Orleans Hotel, New Orleans, United States Duration: 1 Jun 2018 → 6 Jun 2018 http://naacl2018.org/

Conference

Conference	16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Abbreviated title	NAACL HLT 2018
Country/Territory	United States
City	New Orleans
Period	1/06/18 → 6/06/18
Internet address	http://naacl2018.org/

Access to Document

10.18653/v1/N18-2119Licence: Creative Commons: Attribution (CC-BY)

An_Evaluation_of_Image-based_Verb_Prediction_Models_againstAccepted author manuscript, 1.3 MB

https://aclanthology.coli.uni-saarland.de/papers/N18-2119/n18-2119Licence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{a01f503b085e4544871a81fc342274c6,

title = "An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data",

abstract = "Recent research in language and vision has developed models for predicting and disambiguating verbs from images. Here, we ask whether the predictions made by such models correspond to human intuitions about visual verbs. We show that the image regions a verb prediction model identifies as salient for a given verb correlate with the regions fixated by human observers performing a verb classification task.",

author = "Spandana Gella and Frank Keller",

year = "2018",

month = jun,

day = "30",

doi = "10.18653/v1/N18-2119",

language = "English",

pages = "758--763",

booktitle = "The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",

publisher = "Association for Computational Linguistics",

note = "16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL HLT 2018 ; Conference date: 01-06-2018 Through 06-06-2018",

url = "http://naacl2018.org/",

}

Gella, S & Keller, F 2018, An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data. in The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, New Orleans, Louisiana , pp. 758-763, 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Louisiana, United States, 1/06/18. https://doi.org/10.18653/v1/N18-2119

An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data. / Gella, Spandana; Keller, Frank.
The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New Orleans, Louisiana : Association for Computational Linguistics, 2018. p. 758-763.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data

AU - Gella, Spandana

AU - Keller, Frank

PY - 2018/6/30

Y1 - 2018/6/30

N2 - Recent research in language and vision has developed models for predicting and disambiguating verbs from images. Here, we ask whether the predictions made by such models correspond to human intuitions about visual verbs. We show that the image regions a verb prediction model identifies as salient for a given verb correlate with the regions fixated by human observers performing a verb classification task.

AB - Recent research in language and vision has developed models for predicting and disambiguating verbs from images. Here, we ask whether the predictions made by such models correspond to human intuitions about visual verbs. We show that the image regions a verb prediction model identifies as salient for a given verb correlate with the regions fixated by human observers performing a verb classification task.

U2 - 10.18653/v1/N18-2119

DO - 10.18653/v1/N18-2119

M3 - Conference contribution

SP - 758

EP - 763

BT - The 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

PB - Association for Computational Linguistics

CY - New Orleans, Louisiana

T2 - 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

Y2 - 1 June 2018 through 6 June 2018

ER -

An Evaluation of Image-based Verb Prediction Models against Human Eye-tracking Data

Abstract

Conference

Access to Document

Fingerprint

Cite this