Integrating lexical and prosodic features for automatic paragraph segmentation

Catherine Lai, Mireia Farrús, Johanna Moore

Research output: Contribution to journalArticlepeer-review


Spoken documents, such as podcasts or lectures, are a growing presence in everyday life. Being able to automatically identify their discourse structure is an important step to understanding what a spoken document is about. Moreover, finer-grained units, such as paragraphs, are highly desirable for presenting and analyzing spoken content. However, little work has been done on discourse based speech segmentation below the level of broad topics. In order to examine how discourse transitions are cued in speech, we investigate automatic paragraph segmentation of TED talks using lexical and prosodic features. Experiments using Support Vector Machines, AdaBoost, and Neural Networks show that models using supra-sentential prosodic features and induced cue words perform better than those based on the type of lexical cohesion measures often used in broad topic segmentation. Moreover, combining a wide range of individually weak lexical and prosodic predictors improves performance, and modelling contextual information using recurrent neural networks outperforms other approaches by a large margin. Our best results come from using late fusion methods that integrate representations generated by separate lexical and prosodic models while allowing interactions between these features streams rather than treating them as independent information sources. Application to ASR outputs shows that adding prosodic features, particularly using late fusion, can significantly ameliorate decreases in performance due to transcription errors.
Original languageEnglish
Pages (from-to)44-57
JournalSpeech Communication
Early online date11 May 2020
Publication statusPublished - Aug 2020


  • discourse structure
  • paragraph segmentation
  • prosody
  • spoken language understanding
  • coherence
  • speech processing


Dive into the research topics of 'Integrating lexical and prosodic features for automatic paragraph segmentation'. Together they form a unique fingerprint.

Cite this