Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis

Avashna Govender; Cassia Valentini-Botinhao; Simon King

doi:10.21437/SSW.2019-22

Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis

Avashna Govender, Cassia Valentini-Botinhao, Simon King

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Listening to even high quality text-to-speech - such as that generated by a Deep Neural Network (DNN) driving a vocoder - still requires greater cognitive effort than natural speech, under noisy conditions. Vocoding itself, plus errors in predictions of the vocoder speech parameters by the DNN model are assumed to be responsible. To better understand the contribution of each parameter, we construct a range of systems that vary from copysynthesis (i.e., vocoding) to full text-to-speech generated using a Deep Neural Network system. Each system combines some speech parameters (e.g., spectral envelope) from copy-synthesis with other speech parameters (e.g., F0) predicted from text. Cognitive load was measured using a pupillometry paradigm described in our previous work. Our results reveal the differing contributions that each predicted speech parameter makes to increasing cognitive load.

Original language	English
Title of host publication	Proceedings of the 10th ISCA Speech Synthesis Workshop
Publisher	International Speech Communication Association
Pages	121-126
Number of pages	6
DOIs	https://doi.org/10.21437/SSW.2019-22
Publication status	Published - 22 Sept 2019
Event	The 10th ISCA Speech Synthesis Workshop - Austrian museum of folk life and folk art in Vienna, Vienna, Austria Duration: 20 Sept 2019 → 22 Sept 2019 Conference number: 10 http://ssw10.oeaw.ac.at/index.html

Publication series

Name
Publisher	International Speech Communication Association
ISSN (Electronic)	1990-9772

Conference

Conference	The 10th ISCA Speech Synthesis Workshop
Abbreviated title	SSW 2019
Country/Territory	Austria
City	Vienna
Period	20/09/19 → 22/09/19
Internet address	http://ssw10.oeaw.ac.at/index.html

Keywords / Materials (for Non-textual outputs)

text-to-speech
deep neural networks
cognitive load
pupillometry
adverse conditions

Access to Document

10.21437/SSW.2019-22Licence: All Rights Reserved

Measuring the contribution_GOVENDER_DoA020719_AFVAccepted author manuscript, 381 KB

https://www.isca-speech.org/archive/SSW_2019/abstracts/SSW10_O_4-1.htmlLicence: All Rights Reserved

SCRIPT : Speech Synthesis for Spoken Content Production
Yamagishi, J., King, S. & Watts, O.
EPSRC
1/12/16 → 30/11/19
Project: Research

Cite this

@inproceedings{f0070db3262f420fb5399e2c16bb53cd,

title = "Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis",

abstract = "Listening to even high quality text-to-speech - such as that generated by a Deep Neural Network (DNN) driving a vocoder - still requires greater cognitive effort than natural speech, under noisy conditions. Vocoding itself, plus errors in predictions of the vocoder speech parameters by the DNN model are assumed to be responsible. To better understand the contribution of each parameter, we construct a range of systems that vary from copysynthesis (i.e., vocoding) to full text-to-speech generated using a Deep Neural Network system. Each system combines some speech parameters (e.g., spectral envelope) from copy-synthesis with other speech parameters (e.g., F0) predicted from text. Cognitive load was measured using a pupillometry paradigm described in our previous work. Our results reveal the differing contributions that each predicted speech parameter makes to increasing cognitive load. ",

keywords = "text-to-speech, deep neural networks, cognitive load, pupillometry, adverse conditions",

author = "Avashna Govender and Cassia Valentini-Botinhao and Simon King",

year = "2019",

month = sep,

day = "22",

doi = "10.21437/SSW.2019-22",

language = "English",

publisher = "International Speech Communication Association",

pages = "121--126",

booktitle = "Proceedings of the 10th ISCA Speech Synthesis Workshop",

note = "The 10th ISCA Speech Synthesis Workshop, SSW 2019 ; Conference date: 20-09-2019 Through 22-09-2019",

url = "http://ssw10.oeaw.ac.at/index.html",

}

Govender, A, Valentini-Botinhao, C & King, S 2019, Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis. in Proceedings of the 10th ISCA Speech Synthesis Workshop. International Speech Communication Association, pp. 121-126, The 10th ISCA Speech Synthesis Workshop, Vienna, Austria, 20/09/19. https://doi.org/10.21437/SSW.2019-22

Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis. / Govender, Avashna; Valentini-Botinhao, Cassia ; King, Simon.
Proceedings of the 10th ISCA Speech Synthesis Workshop. International Speech Communication Association, 2019. p. 121-126.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis

AU - Govender, Avashna

AU - Valentini-Botinhao, Cassia

AU - King, Simon

N1 - Conference code: 10

PY - 2019/9/22

Y1 - 2019/9/22

N2 - Listening to even high quality text-to-speech - such as that generated by a Deep Neural Network (DNN) driving a vocoder - still requires greater cognitive effort than natural speech, under noisy conditions. Vocoding itself, plus errors in predictions of the vocoder speech parameters by the DNN model are assumed to be responsible. To better understand the contribution of each parameter, we construct a range of systems that vary from copysynthesis (i.e., vocoding) to full text-to-speech generated using a Deep Neural Network system. Each system combines some speech parameters (e.g., spectral envelope) from copy-synthesis with other speech parameters (e.g., F0) predicted from text. Cognitive load was measured using a pupillometry paradigm described in our previous work. Our results reveal the differing contributions that each predicted speech parameter makes to increasing cognitive load.

AB - Listening to even high quality text-to-speech - such as that generated by a Deep Neural Network (DNN) driving a vocoder - still requires greater cognitive effort than natural speech, under noisy conditions. Vocoding itself, plus errors in predictions of the vocoder speech parameters by the DNN model are assumed to be responsible. To better understand the contribution of each parameter, we construct a range of systems that vary from copysynthesis (i.e., vocoding) to full text-to-speech generated using a Deep Neural Network system. Each system combines some speech parameters (e.g., spectral envelope) from copy-synthesis with other speech parameters (e.g., F0) predicted from text. Cognitive load was measured using a pupillometry paradigm described in our previous work. Our results reveal the differing contributions that each predicted speech parameter makes to increasing cognitive load.

KW - text-to-speech

KW - deep neural networks

KW - cognitive load

KW - pupillometry

KW - adverse conditions

U2 - 10.21437/SSW.2019-22

DO - 10.21437/SSW.2019-22

M3 - Conference contribution

SP - 121

EP - 126

BT - Proceedings of the 10th ISCA Speech Synthesis Workshop

PB - International Speech Communication Association

T2 - The 10th ISCA Speech Synthesis Workshop

Y2 - 20 September 2019 through 22 September 2019

ER -

Measuring the contribution to cognitive load of each predicted vocoder speech parameter in DNN-based speech synthesis

Abstract / Description of output

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Projects

SCRIPT : Speech Synthesis for Spoken Content Production

Cite this