Compositional Distributional Semantics with Long Short Term Memory

Phong Le; Willem Zuidema

doi:10.18653/v1/S15-1002

Compositional Distributional Semantics with Long Short Term Memory

Phong Le, Willem Zuidema

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the ‘memory cell’) and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.

Original language	English
Title of host publication	Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics
Place of Publication	Denver, Colorado
Publisher	Association for Computational Linguistics
Pages	10-19
Number of pages	10
DOIs	https://doi.org/10.18653/v1/S15-1002
Publication status	Published - Jun 2015
Event	Fourth Joint Conference on Lexical and Computational Semantics - Denver, United States Duration: 4 Jun 2015 → 5 Jun 2015 https://sites.google.com/site/starsem2015/

Conference

Conference	Fourth Joint Conference on Lexical and Computational Semantics
Abbreviated title	SEM 2015
Country/Territory	United States
City	Denver
Period	4/06/15 → 5/06/15
Internet address	https://sites.google.com/site/starsem2015/

Access to Document

10.18653/v1/S15-1002

http://www.aclweb.org/anthology/S15-1002

Cite this

@inproceedings{f210e258f0bc4b7a9563189cb02fdf95,

title = "Compositional Distributional Semantics with Long Short Term Memory",

abstract = "We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the {\textquoteleft}memory cell{\textquoteright}) and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.",

author = "Phong Le and Willem Zuidema",

year = "2015",

month = jun,

doi = "10.18653/v1/S15-1002",

language = "English",

pages = "10--19",

booktitle = "Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics",

publisher = "Association for Computational Linguistics",

note = "Fourth Joint Conference on Lexical and Computational Semantics , SEM 2015 ; Conference date: 04-06-2015 Through 05-06-2015",

url = "https://sites.google.com/site/starsem2015/",

}

Le, P & Zuidema, W 2015, Compositional Distributional Semantics with Long Short Term Memory. in Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics, Denver, Colorado , pp. 10-19, Fourth Joint Conference on Lexical and Computational Semantics , Denver, Colorado, United States, 4/06/15. https://doi.org/10.18653/v1/S15-1002

TY - GEN

T1 - Compositional Distributional Semantics with Long Short Term Memory

AU - Le, Phong

AU - Zuidema, Willem

PY - 2015/6

Y1 - 2015/6

N2 - We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the ‘memory cell’) and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.

AB - We are proposing an extension of the recursive neural network that makes use of a variant of the long short-term memory architecture. The extension allows information low in parse trees to be stored in a memory register (the ‘memory cell’) and used much later higher up in the parse tree. This provides a solution to the vanishing gradient problem and allows the network to capture long range dependencies. Experimental results show that our composition outperformed the traditional neural-network composition on the Stanford Sentiment Treebank.

U2 - 10.18653/v1/S15-1002

DO - 10.18653/v1/S15-1002

M3 - Conference contribution

SP - 10

EP - 19

BT - Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics

PB - Association for Computational Linguistics

CY - Denver, Colorado

T2 - Fourth Joint Conference on Lexical and Computational Semantics

Y2 - 4 June 2015 through 5 June 2015

ER -

Compositional Distributional Semantics with Long Short Term Memory

Abstract

Conference

Access to Document

Fingerprint

Cite this