Investigation of Maxout Networks for Speech Recognition

P. Swietojanski; J. Li; J-T Huang

doi:10.1109/ICASSP.2014.6855088

Investigation of Maxout Networks for Speech Recognition

P. Swietojanski, J. Li, J-T Huang

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We explore the use of maxout neuron in various aspects of acoustic modelling for large vocabulary speech recognition systems; including low-resource scenario and multilingual knowledge transfers. Through the experiments on voice search and short message dictation datasets, we found that maxout networks are around three times faster to train and offer lower or comparable word error rates on several tasks, when compared to the networks with logistic nonlinearity. We also present a detailed study of the maxout unit internal behaviour suggesting the use of different nonlinearities in different layers.

Original language	English
Title of host publication	Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	7649-7653
Number of pages	5
DOIs	https://doi.org/10.1109/ICASSP.2014.6855088
Publication status	Published - 2014

Access to Document

10.1109/ICASSP.2014.6855088

Swietojanski et al 2014 Investigation of maxoutAccepted author manuscript, 188 KB

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6855088&newsearch=true&queryText=%20Investigation%20of%20maxout%20networks%20for%20speech%20recognition

Cite this

@inproceedings{f31252b30285470582182f94ffa5bd5b,

title = "Investigation of Maxout Networks for Speech Recognition",

abstract = "We explore the use of maxout neuron in various aspects of acoustic modelling for large vocabulary speech recognition systems; including low-resource scenario and multilingual knowledge transfers. Through the experiments on voice search and short message dictation datasets, we found that maxout networks are around three times faster to train and offer lower or comparable word error rates on several tasks, when compared to the networks with logistic nonlinearity. We also present a detailed study of the maxout unit internal behaviour suggesting the use of different nonlinearities in different layers.",

author = "P. Swietojanski and J. Li and J-T Huang",

year = "2014",

doi = "10.1109/ICASSP.2014.6855088",

language = "English",

pages = "7649--7653",

booktitle = "Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

address = "United States",

}

TY - GEN

T1 - Investigation of Maxout Networks for Speech Recognition

AU - Swietojanski, P.

AU - Li, J.

AU - Huang, J-T

PY - 2014

Y1 - 2014

N2 - We explore the use of maxout neuron in various aspects of acoustic modelling for large vocabulary speech recognition systems; including low-resource scenario and multilingual knowledge transfers. Through the experiments on voice search and short message dictation datasets, we found that maxout networks are around three times faster to train and offer lower or comparable word error rates on several tasks, when compared to the networks with logistic nonlinearity. We also present a detailed study of the maxout unit internal behaviour suggesting the use of different nonlinearities in different layers.

AB - We explore the use of maxout neuron in various aspects of acoustic modelling for large vocabulary speech recognition systems; including low-resource scenario and multilingual knowledge transfers. Through the experiments on voice search and short message dictation datasets, we found that maxout networks are around three times faster to train and offer lower or comparable word error rates on several tasks, when compared to the networks with logistic nonlinearity. We also present a detailed study of the maxout unit internal behaviour suggesting the use of different nonlinearities in different layers.

U2 - 10.1109/ICASSP.2014.6855088

DO - 10.1109/ICASSP.2014.6855088

M3 - Conference contribution

SP - 7649

EP - 7653

BT - Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers (IEEE)

ER -

Investigation of Maxout Networks for Speech Recognition

Abstract

Access to Document

Fingerprint

Cite this