Compositional Generalization without Trees using Multiset Tagging and Latent Permutations

Matthias Lindemann, Alexander Koller, Ivan Titov

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

Seq2seq models have been shown to struggle with compositional generalization in semantic parsing, i.e. generalizing to unseen compositions of phenomena that the model handles correctly in isolation.

We phrase semantic parsing as a two-step process: we first tag each input token with a multiset of output tokens. Then we arrange the tokens into an output sequence using a new way of parameterizing and predicting permutations. We formulate predicting a permutation as solving a regularized linear program and we backpropagate through the solver. In contrast to prior work, our approach does not place a priori restrictions on possible permutations, making it very expressive.

Our model outperforms pretrained seq2seq models and prior work on realistic semantic parsing tasks that require generalization to longer examples. We also outperform non-tree-based models on structural generalization on the COGS benchmark. For the first time, we show that a model without an inductive bias provided by trees achieves high accuracy on generalization to deeper recursion.
Original languageEnglish
Title of host publicationProceedings of the 61st Annual Meeting of the Association for Computational Linguistics
Place of PublicationStroudsburg
PublisherAssociation for Computational Linguistics
Pages14488–14506
Number of pages17
Volume1
ISBN (Electronic)9781959429722
Publication statusPublished - 9 Jul 2023
Event61st Annual Meeting of the Association for Computational Linguistics - Toronto, Canada
Duration: 9 Jul 202314 Jul 2023
Conference number: 61
https://2023.aclweb.org/

Conference

Conference61st Annual Meeting of the Association for Computational Linguistics
Abbreviated titleACL 2023
Country/TerritoryCanada
CityToronto
Period9/07/2314/07/23
Internet address

Fingerprint

Dive into the research topics of 'Compositional Generalization without Trees using Multiset Tagging and Latent Permutations'. Together they form a unique fingerprint.

Cite this