Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis

Rasmus Dall; Sandrine  Brognaux; Korin Richmond; Cassia Valentini Botinhao; Gustav Henter; Julia Hirschberg; Junichi Yamagishi; Simon King

doi:10.1109/ICASSP.2016.7472660

Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis

Rasmus Dall, Sandrine Brognaux, Korin Richmond, Cassia Valentini Botinhao, Gustav Henter, Julia Hirschberg, Junichi Yamagishi, Simon King

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Forced alignment for speech synthesis traditionally aligns a phoneme sequence predetermined by the front-end text processing system. This sequence is not altered during alignment, i.e., it is forced, despite possibly being faulty. The consistency assumption is the assumption that these mistakes do not degrade models, as long as the mistakes are consistent across training and synthesis. We present evidence that in the alignment of both standard read prompts and spontaneous speech this phoneme sequence is often wrong, and that this is likely to have a negative impact on acoustic models. A lattice-based forced alignment system allowing for pronunciation variation is implemented, resulting in improved phoneme identity accuracy for both types of speech. A perceptual evaluation of HMM-based voices showed that spontaneous models trained on this improved alignment also improved standard synthesis, despite breaking the consistency assumption

Original language	English
Title of host publication	2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Publisher	Institute of Electrical and Electronics Engineers
Pages	5155-5159
Number of pages	5
ISBN (Print)	978-1-4799-9988-0
DOIs	https://doi.org/10.1109/ICASSP.2016.7472660
Publication status	Published - Mar 2016
Event	41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016 - China, Shanghai, China Duration: 20 Mar 2016 → 25 Mar 2016 https://www2.securecms.com/ICASSP2016/Default.asp

Conference

Conference	41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016
Abbreviated title	ICASSP 2016
Country/Territory	China
City	Shanghai
Period	20/03/16 → 25/03/16
Internet address	https://www2.securecms.com/ICASSP2016/Default.asp

Access to Document

10.1109/ICASSP.2016.7472660

dall2016testing-final-1Accepted author manuscript, 289 KB

http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=7472660

Natural Speech Technology
Renals, S. & King, S.
EPSRC
1/05/11 → 31/07/16
Project: Research

Experiment materials for "Testing the consistency assumption: pronunciation variant forced alignment in read and spontaneous speech synthesis"
Dall, R. (Creator), Edinburgh DataShare, 19 Jan 2016
DOI: 10.7488/ds/1314
Dataset

Cite this

Dall, R., Brognaux, S., Richmond, K., Valentini Botinhao, C., Henter, G., Hirschberg, J., Yamagishi, J., & King, S. (2016). Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 5155-5159). Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICASSP.2016.7472660

@inproceedings{f9fb25b69e2949308f0da6e60cb42dd5,

title = "Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis",

abstract = "Forced alignment for speech synthesis traditionally aligns a phoneme sequence predetermined by the front-end text processing system. This sequence is not altered during alignment, i.e., it is forced, despite possibly being faulty. The consistency assumption is the assumption that these mistakes do not degrade models, as long as the mistakes are consistent across training and synthesis. We present evidence that in the alignment of both standard read prompts and spontaneous speech this phoneme sequence is often wrong, and that this is likely to have a negative impact on acoustic models. A lattice-based forced alignment system allowing for pronunciation variation is implemented, resulting in improved phoneme identity accuracy for both types of speech. A perceptual evaluation of HMM-based voices showed that spontaneous models trained on this improved alignment also improved standard synthesis, despite breaking the consistency assumption",

author = "Rasmus Dall and Sandrine Brognaux and Korin Richmond and {Valentini Botinhao}, Cassia and Gustav Henter and Julia Hirschberg and Junichi Yamagishi and Simon King",

year = "2016",

month = mar,

doi = "10.1109/ICASSP.2016.7472660",

language = "English",

isbn = "978-1-4799-9988-0",

pages = "5155--5159",

booktitle = "2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)",

publisher = "Institute of Electrical and Electronics Engineers",

address = "United States",

note = "41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016, ICASSP 2016 ; Conference date: 20-03-2016 Through 25-03-2016",

url = "https://www2.securecms.com/ICASSP2016/Default.asp",

}

Dall, R, Brognaux, S, Richmond, K , Valentini Botinhao, C, Henter, G, Hirschberg, J, Yamagishi, J & King, S 2016, Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis. in 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers, pp. 5155-5159, 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016, Shanghai, China, 20/03/16. https://doi.org/10.1109/ICASSP.2016.7472660

Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis. / Dall, Rasmus; Brognaux, Sandrine ; Richmond, Korin et al.
2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers, 2016. p. 5155-5159.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis

AU - Dall, Rasmus

AU - Brognaux, Sandrine

AU - Richmond, Korin

AU - Valentini Botinhao, Cassia

AU - Henter, Gustav

AU - Hirschberg, Julia

AU - Yamagishi, Junichi

AU - King, Simon

PY - 2016/3

Y1 - 2016/3

N2 - Forced alignment for speech synthesis traditionally aligns a phoneme sequence predetermined by the front-end text processing system. This sequence is not altered during alignment, i.e., it is forced, despite possibly being faulty. The consistency assumption is the assumption that these mistakes do not degrade models, as long as the mistakes are consistent across training and synthesis. We present evidence that in the alignment of both standard read prompts and spontaneous speech this phoneme sequence is often wrong, and that this is likely to have a negative impact on acoustic models. A lattice-based forced alignment system allowing for pronunciation variation is implemented, resulting in improved phoneme identity accuracy for both types of speech. A perceptual evaluation of HMM-based voices showed that spontaneous models trained on this improved alignment also improved standard synthesis, despite breaking the consistency assumption

AB - Forced alignment for speech synthesis traditionally aligns a phoneme sequence predetermined by the front-end text processing system. This sequence is not altered during alignment, i.e., it is forced, despite possibly being faulty. The consistency assumption is the assumption that these mistakes do not degrade models, as long as the mistakes are consistent across training and synthesis. We present evidence that in the alignment of both standard read prompts and spontaneous speech this phoneme sequence is often wrong, and that this is likely to have a negative impact on acoustic models. A lattice-based forced alignment system allowing for pronunciation variation is implemented, resulting in improved phoneme identity accuracy for both types of speech. A perceptual evaluation of HMM-based voices showed that spontaneous models trained on this improved alignment also improved standard synthesis, despite breaking the consistency assumption

U2 - 10.1109/ICASSP.2016.7472660

DO - 10.1109/ICASSP.2016.7472660

M3 - Conference contribution

SN - 978-1-4799-9988-0

SP - 5155

EP - 5159

BT - 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

PB - Institute of Electrical and Electronics Engineers

T2 - 41st IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2016

Y2 - 20 March 2016 through 25 March 2016

ER -

Dall R, Brognaux S, Richmond K , Valentini Botinhao C, Henter G, Hirschberg J et al. Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis. In 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Institute of Electrical and Electronics Engineers. 2016. p. 5155-5159 doi: 10.1109/ICASSP.2016.7472660

Testing the Consistency Assumption: Pronunciation Variant Forced Alignment in Read and Spontaneous Speech Synthesis

Abstract / Description of output

Conference

Access to Document

Fingerprint

Projects

Natural Speech Technology

Datasets

Experiment materials for "Testing the consistency assumption: pronunciation variant forced alignment in read and spontaneous speech synthesis"

Cite this