Automatic Transformation of a Video Using Multimodal Information for an Engaging Exploration Experience

Fahim A Salim, Fasih Haider, Saturnino Luz, Owen Conlan

Research output: Contribution to journalArticlepeer-review


Exploring the content of a video is typically inefficient due to the linear streamed nature of its media and the lack of interactivity. While different approaches have been proposed for enhancing the exploration experience of video content, the general view of video content has remained basically the same, that is, a continuous stream of images. It is our contention that such a conservative view on video limits its potential value as a content source. This paper presents An Alternative Representation of Video via feature Extraction (RAAVE), a novel approach to transform videos from a linear stream of content into an adaptive interactive multimedia document and thereby enhance the exploration potential of video content by providing a more engaging user experience. We explore the idea of viewing video as a diverse multimedia content source, opening new opportunities and applications to explore and consume video content. A modular framework and algorithm for the representation engine and template collection is described. The representation engine based approach is evaluated through development of a prototype system grounded on the design of the proposed approach, allowing users to perform multiple content exploration tasks within a video. The evaluation demonstrated RAAVE’s ability to provide users with a more engaging, efficient and effective experience than a typical multimedia player while performing video exploration tasks
Original languageUndefined/Unknown
JournalApplied Sciences
Issue number9
Publication statusPublished - 27 Apr 2020

Cite this