Factorizing Content and Budget Decisions in Abstractive Summarization of Long Documents

Marcio Fernandes De Moraes Fonseca, Yftah Ziser, Shay B Cohen

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We argue that disentangling content selection from the budget used to cover salient content improves the performance and applicability of abstractive summarizers. Our method, FactorSum, does this disentanglement by factorizing summarization into two steps through an energy function: (1) generation of abstractive summary views covering salient information in subsets of the input document (document views); (2) combination of these views into a final summary, following a budget and content guidance. This guidance may come from different sources, including from an advisor model such as BART or BigBird, or in oracle mode – from the reference. This factorization achieves significantly higher ROUGE scores on multiple benchmarks for long document summarization, namely PubMed, arXiv, and GovReport. Most notably, our model is effective for domain adaptation. When trained only on PubMed samples, it achieves a 46.29 ROUGE-1 score on arXiv, outperforming PEGASUS trained in domain by a large margin. Our experimental results indicate that the performance gains are due to more flexible budget adaptation and processing of shorter contexts provided by partial document views.
Original languageEnglish
Title of host publicationProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing
EditorsYoav Goldberg, Zornitsa Kozareva, Yue Zhang
Place of PublicationAbu Dhabi
PublisherAssociation for Computational Linguistics (ACL)
Pages6341–6364
Number of pages25
Publication statusPublished - 2 Feb 2023
EventThe 2022 Conference on Empirical Methods in Natural Language Processing - Abu Dhabi, United Arab Emirates
Duration: 7 Dec 202211 Dec 2022
https://2022.emnlp.org/

Conference

ConferenceThe 2022 Conference on Empirical Methods in Natural Language Processing
Abbreviated titleEMNLP 2022
Country/TerritoryUnited Arab Emirates
CityAbu Dhabi
Period7/12/2211/12/22
Internet address

Fingerprint

Dive into the research topics of 'Factorizing Content and Budget Decisions in Abstractive Summarization of Long Documents'. Together they form a unique fingerprint.

Cite this