Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating

Julie-Anne Meaney; Steve R. Wilson; Walid Magdy

Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating

Julie-Anne Meaney, Steve R. Wilson, Walid Magdy

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

The use of pre-trained language models such as BERT and ULMFiT has become increasingly popular in shared tasks, due to their powerful language modelling capabilities. Our entry to SemEval uses ERNIE 2.0, a language model which is pre-trained on a large number of tasks to enrich the semantic and syntactic information learned. ERNIE’s knowledge masking pre-training task is a unique method for learning about named entities, and we hypothesise that it may be of use in a dataset which is built on news headlines and which contains many named entities. We optimize the hyperparameters in a regression and classification model and find that the hyperparameters we selected helped to make bigger gains in the classification model than the regression model.

Original language	English
Title of host publication	Proceedings of the Fourteenth Workshop on Semantic Evaluation
Publisher	Association for Computational Linguistics
Pages	1049–1054
Number of pages	6
ISBN (Print)	978-1-952148-31-6
Publication status	Published - 12 Dec 2020
Event	International Workshop on Semantic Evaluation 2020 - Barcelona, Spain Duration: 12 Dec 2020 → 13 Dec 2020 http://alt.qcri.org/semeval2020/#

Workshop

Workshop	International Workshop on Semantic Evaluation 2020
Abbreviated title	SemEval 2020
Country/Territory	Spain
City	Barcelona
Period	12/12/20 → 13/12/20
Internet address	http://alt.qcri.org/semeval2020/#

Access to Document

Smash at SemEval_MEANEY_DOA24062020_VOR_CC-BYFinal published version, 140 KBLicence: Creative Commons: Attribution (CC-BY)

https://www.aclweb.org/anthology/2020.semeval-1.137Licence: Creative Commons: Attribution (CC-BY)

Cite this

@inproceedings{7ebba9c814854e068bf36be4e1c04849,

title = "Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating",

abstract = "The use of pre-trained language models such as BERT and ULMFiT has become increasingly popular in shared tasks, due to their powerful language modelling capabilities. Our entry to SemEval uses ERNIE 2.0, a language model which is pre-trained on a large number of tasks to enrich the semantic and syntactic information learned. ERNIE{\textquoteright}s knowledge masking pre-training task is a unique method for learning about named entities, and we hypothesise that it may be of use in a dataset which is built on news headlines and which contains many named entities. We optimize the hyperparameters in a regression and classification model and find that the hyperparameters we selected helped to make bigger gains in the classification model than the regression model.",

author = "Julie-Anne Meaney and Wilson, {Steve R.} and Walid Magdy",

year = "2020",

month = dec,

day = "12",

language = "English",

isbn = "978-1-952148-31-6",

pages = "1049–1054",

booktitle = "Proceedings of the Fourteenth Workshop on Semantic Evaluation",

publisher = "Association for Computational Linguistics",

note = "International Workshop on Semantic Evaluation 2020, SemEval 2020 ; Conference date: 12-12-2020 Through 13-12-2020",

url = "http://alt.qcri.org/semeval2020/#",

}

Meaney, J-A, Wilson, SR & Magdy, W 2020, Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating. in Proceedings of the Fourteenth Workshop on Semantic Evaluation. Association for Computational Linguistics, pp. 1049–1054, International Workshop on Semantic Evaluation 2020, Barcelona, Spain, 12/12/20. <https://www.aclweb.org/anthology/2020.semeval-1.137>

Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating. / Meaney, Julie-Anne; Wilson, Steve R.; Magdy, Walid.
Proceedings of the Fourteenth Workshop on Semantic Evaluation. Association for Computational Linguistics, 2020. p. 1049–1054.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating

AU - Meaney, Julie-Anne

AU - Wilson, Steve R.

AU - Magdy, Walid

PY - 2020/12/12

Y1 - 2020/12/12

N2 - The use of pre-trained language models such as BERT and ULMFiT has become increasingly popular in shared tasks, due to their powerful language modelling capabilities. Our entry to SemEval uses ERNIE 2.0, a language model which is pre-trained on a large number of tasks to enrich the semantic and syntactic information learned. ERNIE’s knowledge masking pre-training task is a unique method for learning about named entities, and we hypothesise that it may be of use in a dataset which is built on news headlines and which contains many named entities. We optimize the hyperparameters in a regression and classification model and find that the hyperparameters we selected helped to make bigger gains in the classification model than the regression model.

AB - The use of pre-trained language models such as BERT and ULMFiT has become increasingly popular in shared tasks, due to their powerful language modelling capabilities. Our entry to SemEval uses ERNIE 2.0, a language model which is pre-trained on a large number of tasks to enrich the semantic and syntactic information learned. ERNIE’s knowledge masking pre-training task is a unique method for learning about named entities, and we hypothesise that it may be of use in a dataset which is built on news headlines and which contains many named entities. We optimize the hyperparameters in a regression and classification model and find that the hyperparameters we selected helped to make bigger gains in the classification model than the regression model.

M3 - Conference contribution

SN - 978-1-952148-31-6

SP - 1049

EP - 1054

BT - Proceedings of the Fourteenth Workshop on Semantic Evaluation

PB - Association for Computational Linguistics

T2 - International Workshop on Semantic Evaluation 2020

Y2 - 12 December 2020 through 13 December 2020

ER -

Smash at SemEval-2020 Task 7: Optimizing the Hyperparameters of ERNIE 2.0 for Humor Ranking and Rating

Abstract / Description of output

Workshop

Access to Document

Fingerprint

Cite this