Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks

Lauri Juvela; Bajibabu Bollepalli; Xin Wang; Hirokazu Kameoka; Manu Airaksinen; Junichi Yamagishi; Paavo Alku

doi:10.1109/ICASSP.2018.8461852

Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks

Lauri Juvela, Bajibabu Bollepalli, Xin Wang, Hirokazu Kameoka, Manu Airaksinen, Junichi Yamagishi, Paavo Alku

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitchsynchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network -based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.
Index Terms— MFCC, Pitch prediction, Mel-filterbank inversion, Excitation modeling, Generative adversarial networks

Original language	English
Title of host publication	2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Subtitle of host publication	Calgary, AB, Canada
Place of Publication	Calgary, Alberta, Canada
Publisher	Institute of Electrical and Electronics Engineers (IEEE)
Pages	5679-5683
Number of pages	5
ISBN (Electronic)	978-1-5386-4658-8
ISBN (Print)	978-1-5386-4659-5
DOIs	https://doi.org/10.1109/ICASSP.2018.8461852
Publication status	Published - 13 Sep 2018
Event	2018 IEEE International Conference on Acoustics, Speech and Signal Processing - Calgary, Canada Duration: 15 Apr 2018 → 20 Apr 2018 https://2018.ieeeicassp.org/ https://2018.ieeeicassp.org/default.asp

Publication series

Name
Publisher	IEEE
ISSN (Electronic)	2379-190X

Conference

Conference	2018 IEEE International Conference on Acoustics, Speech and Signal Processing
Abbreviated title	ICASSP 2018
Country/Territory	Canada
City	Calgary
Period	15/04/18 → 20/04/18
Internet address	https://2018.ieeeicassp.org/ https://2018.ieeeicassp.org/default.asp

Access to Document

10.1109/ICASSP.2018.8461852

SPEECH WAVEFORM SYNTHESIS FROM MFCC SEQUENCES WITH GENERATIVEAccepted author manuscript, 266 KB

Cite this

Juvela, L., Bollepalli, B., Wang, X., Kameoka, H., Airaksinen, M., Yamagishi, J., & Alku, P. (2018). Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada (pp. 5679-5683). Institute of Electrical and Electronics Engineers (IEEE). https://doi.org/10.1109/ICASSP.2018.8461852

@inproceedings{e38d5da6ca0f4c929d0e4bd4c33df668,

title = "Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks",

abstract = "This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitchsynchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network -based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.Index Terms— MFCC, Pitch prediction, Mel-filterbank inversion, Excitation modeling, Generative adversarial networks",

author = "Lauri Juvela and Bajibabu Bollepalli and Xin Wang and Hirokazu Kameoka and Manu Airaksinen and Junichi Yamagishi and Paavo Alku",

year = "2018",

month = sep,

day = "13",

doi = "10.1109/ICASSP.2018.8461852",

language = "English",

isbn = "978-1-5386-4659-5",

publisher = "Institute of Electrical and Electronics Engineers (IEEE)",

pages = "5679--5683",

booktitle = "2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

address = "United States",

note = "2018 IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2018 ; Conference date: 15-04-2018 Through 20-04-2018",

url = "https://2018.ieeeicassp.org/, https://2018.ieeeicassp.org/default.asp",

}

Juvela, L, Bollepalli, B, Wang, X, Kameoka, H, Airaksinen, M, Yamagishi, J & Alku, P 2018, Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks. in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada. Institute of Electrical and Electronics Engineers (IEEE), Calgary, Alberta, Canada, pp. 5679-5683, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, Canada, 15/04/18. https://doi.org/10.1109/ICASSP.2018.8461852

Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks. / Juvela, Lauri; Bollepalli, Bajibabu; Wang, Xin et al.

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada. Calgary, Alberta, Canada : Institute of Electrical and Electronics Engineers (IEEE), 2018. p. 5679-5683.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks

AU - Juvela, Lauri

AU - Bollepalli, Bajibabu

AU - Wang, Xin

AU - Kameoka, Hirokazu

AU - Airaksinen, Manu

AU - Yamagishi, Junichi

AU - Alku, Paavo

PY - 2018/9/13

Y1 - 2018/9/13

N2 - This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitchsynchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network -based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.Index Terms— MFCC, Pitch prediction, Mel-filterbank inversion, Excitation modeling, Generative adversarial networks

AB - This paper proposes a method for generating speech from filterbank mel frequency cepstral coefficients (MFCC), which are widely used in speech applications, such as ASR, but are generally considered unusable for speech synthesis. First, we predict fundamental frequency and voicing information from MFCCs with an autoregressive recurrent neural net. Second, the spectral envelope information contained in MFCCs is converted to all-pole filters, and a pitchsynchronous excitation model matched to these filters is trained. Finally, we introduce a generative adversarial network -based noise model to add a realistic high-frequency stochastic component to the modeled excitation signal. The results show that high quality speech reconstruction can be obtained, given only MFCC information at test time.Index Terms— MFCC, Pitch prediction, Mel-filterbank inversion, Excitation modeling, Generative adversarial networks

U2 - 10.1109/ICASSP.2018.8461852

DO - 10.1109/ICASSP.2018.8461852

M3 - Conference contribution

SN - 978-1-5386-4659-5

SP - 5679

EP - 5683

BT - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers (IEEE)

CY - Calgary, Alberta, Canada

T2 - 2018 IEEE International Conference on Acoustics, Speech and Signal Processing

Y2 - 15 April 2018 through 20 April 2018

ER -

Juvela L, Bollepalli B, Wang X, Kameoka H, Airaksinen M, Yamagishi J et al. Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks. In 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP): Calgary, AB, Canada. Calgary, Alberta, Canada: Institute of Electrical and Electronics Engineers (IEEE). 2018. p. 5679-5683 doi: 10.1109/ICASSP.2018.8461852

Speech Waveform Synthesis From MFCC Sequences With Generative Adversarial Networks

Abstract

Publication series

Conference

Access to Document

Fingerprint

Cite this