Finding the Right Moment: Human-Assisted Trailer Creation Via Task Composition

Pinelopi Papalampidi, Frank Keller, Mirella Lapata

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

Movie trailers perform multiple functions: they introduce viewers to the story, convey the mood and artistic style of the film, and encourage audiences to see the movie. These diverse functions make trailer creation a challenging endeavor. In this work, we focus on finding trailer moments in a movie, i.e., shots that could be potentially included in a trailer. We decompose this task into two subtasks: narrative structure identification and sentiment prediction. We model movies as graphs, where nodes are shots and edges denote semantic relations between them. We learn these relations using joint contrastive training which distills rich textual information (e.g., characters, actions, situations) from screenplays. An unsupervised algorithm then traverses the graph and selects trailer moments from the movie that human judges prefer to ones selected by competitive supervised approaches. A main advantage of our algorithm is that it uses interpretable criteria, which allows us to deploy it in an interactive tool for trailer creation with a human in the loop. Our tool allows users to select trailer shots in under 30 minutes that are superior to fully automatic methods and comparable to (exclusive) manual selection by experts.
Original languageEnglish
Pages (from-to)292-304
Number of pages12
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Issue number1
Publication statusPublished - 5 Dec 2023

Keywords / Materials (for Non-textual outputs)

  • Natural language processing
  • computer vision
  • machine learning
  • neural networks
  • video
  • user interaction


Dive into the research topics of 'Finding the Right Moment: Human-Assisted Trailer Creation Via Task Composition'. Together they form a unique fingerprint.

Cite this