Hypothesis Only Baselines in Natural Language Inference

Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger, Benjamin Van Durme

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI). Especially when an NLI dataset assumes inference is occurring based purely on the relationship between a context and a hypothesis, it follows that assessing entailment relations while ignoring the provided context is a degenerate solution. Yet, through experiments on 10 distinct NLI datasets, we find that this approach, which we refer to as a hypothesis-only model, is able to significantly outperform a majority-class baseline across a number of NLI datasets. Our analysis suggests that statistical irregularities may allow a model to perform NLI in some datasets beyond what should be achievable without access to the context.
Original languageEnglish
Title of host publicationProceedings of the Seventh Joint Conference on Lexical and Computational Semantics
Place of PublicationNew Orleans, Louisiana
PublisherAssociation for Computational Linguistics
Pages180-191
Number of pages12
DOIs
Publication statusPublished - 1 Jun 2018
Event7th Seventh Joint Conference on Lexical and Computational Semantics - New Orleans, United States
Duration: 5 Jun 20186 Jun 2018
Conference number: 7
https://sites.google.com/view/starsem2018/

Conference

Conference7th Seventh Joint Conference on Lexical and Computational Semantics
Country/TerritoryUnited States
CityNew Orleans
Period5/06/186/06/18
Internet address

Fingerprint

Dive into the research topics of 'Hypothesis Only Baselines in Natural Language Inference'. Together they form a unique fingerprint.

Cite this