Regularization of context-dependent deep neural networks with context-independent multi-task training

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

The use of context-dependent targets has become standard in hybrid DNN systems for automatic speech recognition. However, we argue that despite the use of state-tying, optimising to context-dependent targets can lead to over-fitting, and that discriminating between arbitrary tied context-dependent targets may not be optimal. We propose a multitask learning method where the network jointly predicts context-dependent and monophone targets. We evaluate the method on a large-vocabulary lecture recognition task and show that it yields relative improvements of 3-10% over baseline systems.
Original languageEnglish
Title of host publicationProc IEEE International Conference on Acoustics, Speech and Signal Processing
Place of PublicationBrisbane, QLD, Australia
PublisherInstitute of Electrical and Electronics Engineers
Pages4290-4294
Number of pages5
ISBN (Electronic)978-1-4673-6997-8
DOIs
Publication statusPublished - 6 Aug 2015
Event40th IEEE International Conference on Acoustics, Speech and Signal Processing - Brisbane Convention & Exhibition Centre, Brisbane, Australia
Duration: 19 Apr 201524 Apr 2015

Conference

Conference40th IEEE International Conference on Acoustics, Speech and Signal Processing
Abbreviated titleICASSP 2015
Country/TerritoryAustralia
CityBrisbane
Period19/04/1524/04/15

Fingerprint

Dive into the research topics of 'Regularization of context-dependent deep neural networks with context-independent multi-task training'. Together they form a unique fingerprint.

Cite this