Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019

Kellie Webster; Marta R. Costa-jussà; Christian Hardmeier; Will Radford

doi:10.18653/v1/W19-3801

Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019

Kellie Webster, Marta R. Costa-jussà, Christian Hardmeier, Will Radford

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

The 1st ACL workshop on Gender Bias in Natural Language Processing included a shared task on gendered ambiguous pronoun (GAP) resolution. This task was based on the coreference challenge defined in Webster et al. (2018), designed to benchmark the ability of systems to resolve pronouns in real-world contexts in a gender-fair way. 263 teams competed via a Kaggle competition, with the winning system achieving logloss of 0.13667 and near gender parity. We review the approaches of eleven systems with accepted description papers, noting their effective use of BERT (Devlin et al., 2018), both via fine-tuning and for feature extraction, as well as ensembling.

Original language	English
Title of host publication	Proceedings of the First Workshop on Gender Bias in Natural Language Processing
Place of Publication	Florence, Italy
Publisher	Association for Computational Linguistics
Pages	1-7
Number of pages	7
ISBN (Electronic)	978-1-950737-40-6
DOIs	https://doi.org/10.18653/v1/W19-3801
Publication status	Published - 2 Aug 2019
Event	57th Annual Meeting of the Association for Computational Linguistics - Fortezza da Basso, Florence, Italy Duration: 28 Jul 2019 → 2 Aug 2019 Conference number: 57 http://www.acl2019.org/EN/index.xhtml

Conference

Conference	57th Annual Meeting of the Association for Computational Linguistics
Abbreviated title	ACL 2019
Country/Territory	Italy
City	Florence
Period	28/07/19 → 2/08/19
Internet address	http://www.acl2019.org/EN/index.xhtml

Access to Document

10.18653/v1/W19-3801Licence: Creative Commons: Attribution (CC-BY)

Gendered Ambiguous Pronoun_WEBSTER_DOA13052019_VOR_CC-BYFinal published version, 208 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/W19-3801Licence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{edf845baa9aa4a62a6d608c24e4a7e78,

title = "Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019",

abstract = "The 1st ACL workshop on Gender Bias in Natural Language Processing included a shared task on gendered ambiguous pronoun (GAP) resolution. This task was based on the coreference challenge defined in Webster et al. (2018), designed to benchmark the ability of systems to resolve pronouns in real-world contexts in a gender-fair way. 263 teams competed via a Kaggle competition, with the winning system achieving logloss of 0.13667 and near gender parity. We review the approaches of eleven systems with accepted description papers, noting their effective use of BERT (Devlin et al., 2018), both via fine-tuning and for feature extraction, as well as ensembling.",

author = "Kellie Webster and Costa-juss{\`a}, {Marta R.} and Christian Hardmeier and Will Radford",

year = "2019",

month = aug,

day = "2",

doi = "10.18653/v1/W19-3801",

language = "English",

pages = "1--7",

booktitle = "Proceedings of the First Workshop on Gender Bias in Natural Language Processing",

publisher = "Association for Computational Linguistics",

note = "57th Annual Meeting of the Association for Computational Linguistics, ACL 2019 ; Conference date: 28-07-2019 Through 02-08-2019",

url = "http://www.acl2019.org/EN/index.xhtml",

}

Webster, K, Costa-jussà, MR, Hardmeier, C & Radford, W 2019, Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019. in Proceedings of the First Workshop on Gender Bias in Natural Language Processing. Association for Computational Linguistics, Florence, Italy, pp. 1-7, 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy, 28/07/19. https://doi.org/10.18653/v1/W19-3801

Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019. / Webster, Kellie; Costa-jussà, Marta R.; Hardmeier, Christian et al.
Proceedings of the First Workshop on Gender Bias in Natural Language Processing. Florence, Italy: Association for Computational Linguistics, 2019. p. 1-7.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019

AU - Webster, Kellie

AU - Costa-jussà, Marta R.

AU - Hardmeier, Christian

AU - Radford, Will

N1 - Conference code: 57

PY - 2019/8/2

Y1 - 2019/8/2

N2 - The 1st ACL workshop on Gender Bias in Natural Language Processing included a shared task on gendered ambiguous pronoun (GAP) resolution. This task was based on the coreference challenge defined in Webster et al. (2018), designed to benchmark the ability of systems to resolve pronouns in real-world contexts in a gender-fair way. 263 teams competed via a Kaggle competition, with the winning system achieving logloss of 0.13667 and near gender parity. We review the approaches of eleven systems with accepted description papers, noting their effective use of BERT (Devlin et al., 2018), both via fine-tuning and for feature extraction, as well as ensembling.

AB - The 1st ACL workshop on Gender Bias in Natural Language Processing included a shared task on gendered ambiguous pronoun (GAP) resolution. This task was based on the coreference challenge defined in Webster et al. (2018), designed to benchmark the ability of systems to resolve pronouns in real-world contexts in a gender-fair way. 263 teams competed via a Kaggle competition, with the winning system achieving logloss of 0.13667 and near gender parity. We review the approaches of eleven systems with accepted description papers, noting their effective use of BERT (Devlin et al., 2018), both via fine-tuning and for feature extraction, as well as ensembling.

U2 - 10.18653/v1/W19-3801

DO - 10.18653/v1/W19-3801

M3 - Conference contribution

SP - 1

EP - 7

BT - Proceedings of the First Workshop on Gender Bias in Natural Language Processing

PB - Association for Computational Linguistics

CY - Florence, Italy

T2 - 57th Annual Meeting of the Association for Computational Linguistics

Y2 - 28 July 2019 through 2 August 2019

ER -

Gendered Ambiguous Pronoun (GAP) Shared Task at the Gender Bias in NLP Workshop 2019

Abstract / Description of output

Conference

Access to Document

Fingerprint

Cite this