Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features

Ondrej Klejch; Peter Bell; Steve Renals

doi:10.1109/ICASSP.2017.7953248

Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

In this paper we present an extension of our previously described neural machine translation based system for punctuated transcription. This extension allows the system to map from per frame acoustic features to word level representations by replacing the traditional encoder in the encoder-decoder architecture with a hierarchical encoder. Furthermore, we show that a system combining lexical and acoustic features significantly outperforms systems using only a single source
of features on all measured punctuation marks. The combination of lexical and acoustic features achieves a significant improvement in F-Measure of 1.5 absolute over the purely lexical neural machine translation based system.

Original language	English
Title of host publication	The 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	5700-5704
Number of pages	5
ISBN (Print)	978-1-5090-4117-6
DOIs	https://doi.org/10.1109/ICASSP.2017.7953248
Publication status	Published - 19 Jun 2017
Event	42nd IEEE International Conference on Acoustics, Speech and Signal Processing - New Orleans, United States Duration: 5 Mar 2017 → 9 Mar 2017 http://www.ieee-icassp2017.org/

Publication series

Name
Publisher	IEEE
ISSN (Electronic)	2379-190X

Conference

Conference	42nd IEEE International Conference on Acoustics, Speech and Signal Processing
Abbreviated title	ICASSP 2017
Country/Territory	United States
City	New Orleans
Period	5/03/17 → 9/03/17
Internet address	http://www.ieee-icassp2017.org/

Access to Document

10.1109/ICASSP.2017.7953248

icassp-2017-1Accepted author manuscript, 1.65 MB

http://ieeexplore.ieee.org/document/7953248/

1 Finished

SUMMA - Scalable Understanding of Mulitingual Media
Renals, S., Birch-Mayne, A. & Cohen, S.
EU government bodies
1/02/16 → 31/01/19
Project: Research

Cite this

@inproceedings{3227e8cb3b4448548f4054f78fc0ba9f,

title = "Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features",

abstract = "In this paper we present an extension of our previously described neural machine translation based system for punctuated transcription. This extension allows the system to map from per frame acoustic features to word level representations by replacing the traditional encoder in the encoder-decoder architecture with a hierarchical encoder. Furthermore, we show that a system combining lexical and acoustic features significantly outperforms systems using only a single sourceof features on all measured punctuation marks. The combination of lexical and acoustic features achieves a significant improvement in F-Measure of 1.5 absolute over the purely lexical neural machine translation based system.",

author = "Ondrej Klejch and Peter Bell and Steve Renals",

year = "2017",

month = jun,

day = "19",

doi = "10.1109/ICASSP.2017.7953248",

language = "English",

isbn = "978-1-5090-4117-6",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "5700--5704",

booktitle = "The 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017)",

address = "United States",

note = "42nd IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2017 ; Conference date: 05-03-2017 Through 09-03-2017",

url = "http://www.ieee-icassp2017.org/",

}

Klejch, O , Bell, P & Renals, S 2017, Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features. in The 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017). Institute of Electrical and Electronics Engineers (IEEE), pp. 5700-5704, 42nd IEEE International Conference on Acoustics, Speech and Signal Processing, New Orleans, Louisiana, United States, 5/03/17. https://doi.org/10.1109/ICASSP.2017.7953248

Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features. / Klejch, Ondrej ; Bell, Peter; Renals, Steve.

The 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017). Institute of Electrical and Electronics Engineers (IEEE), 2017. p. 5700-5704.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features

AU - Klejch, Ondrej

AU - Bell, Peter

AU - Renals, Steve

PY - 2017/6/19

Y1 - 2017/6/19

N2 - In this paper we present an extension of our previously described neural machine translation based system for punctuated transcription. This extension allows the system to map from per frame acoustic features to word level representations by replacing the traditional encoder in the encoder-decoder architecture with a hierarchical encoder. Furthermore, we show that a system combining lexical and acoustic features significantly outperforms systems using only a single sourceof features on all measured punctuation marks. The combination of lexical and acoustic features achieves a significant improvement in F-Measure of 1.5 absolute over the purely lexical neural machine translation based system.

AB - In this paper we present an extension of our previously described neural machine translation based system for punctuated transcription. This extension allows the system to map from per frame acoustic features to word level representations by replacing the traditional encoder in the encoder-decoder architecture with a hierarchical encoder. Furthermore, we show that a system combining lexical and acoustic features significantly outperforms systems using only a single sourceof features on all measured punctuation marks. The combination of lexical and acoustic features achieves a significant improvement in F-Measure of 1.5 absolute over the purely lexical neural machine translation based system.

U2 - 10.1109/ICASSP.2017.7953248

DO - 10.1109/ICASSP.2017.7953248

M3 - Conference contribution

SN - 978-1-5090-4117-6

SP - 5700

EP - 5704

BT - The 42nd IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2017)

PB - Institute of Electrical and Electronics Engineers (IEEE)

T2 - 42nd IEEE International Conference on Acoustics, Speech and Signal Processing

Y2 - 5 March 2017 through 9 March 2017

ER -

Sequence-to-Sequence Models for Punctuated Transcription Combing Lexical and Acoustic Features

Abstract

Publication series

Conference

Access to Document

Fingerprint

Projects

SUMMA - Scalable Understanding of Mulitingual Media

Cite this