Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education

Xingran Ruan; Charaka Palansuriya; Aurora Constantin

doi:10.1007/978-3-031-36272-9_70

Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education

Xingran Ruan, Charaka Palansuriya, Aurora Constantin

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract / Description of output

Facial expressions of learners are relevant to their learning outcomes. The recognition of their emotional status influences the benefits of instruction or feedback provided by the intelligent tutor in education. However, learners’ emotions expressed during interactions with the intelligent tutor are mostly detected by self-reports of learners or judges who observe them in manually. The automated Facial Emotion Recognition (FER) task has been a challenging problem for intelligent tutors. The state-of-art automated FER methods target six basic emotions instead of learning-related emotions (e.g., neutral, confused, frustrated, and bored). Thus our research contributes to training a machine learning (ML) model to recognise learning-related emotions for intelligent tutors automatically, based on an Affective Dynamics (AD) model. We implement the AD model into our loss function (AD-loss) to fine tune the ML model. In the test scenario, the AD-loss method improves the performance of state-of-art FER algorithms.

Original language	English
Title of host publication	Artificial Intelligence in Education
Subtitle of host publication	24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings
Publisher	Springer
Pages	774-779
Number of pages	6
Volume	13916
ISBN (Electronic)	9783031362729
ISBN (Print)	9783031362712
DOIs	https://doi.org/10.1007/978-3-031-36272-9_70
Publication status	Published - 25 Jun 2023
Event	24th International Conference on Artificial Intelligence in Education - Tokyo, Japan Duration: 3 Jul 2023 → 7 Jul 2023 Conference number: 24 https://www.aied2023.org/

Publication series

Name	Lecture Notes in Computer Science
Publisher	Springer
Volume	13916
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	24th International Conference on Artificial Intelligence in Education
Abbreviated title	AIED 2023
Country/Territory	Japan
City	Tokyo
Period	3/07/23 → 7/07/23
Internet address	https://www.aied2023.org/

Keywords / Materials (for Non-textual outputs)

facial emotion recognition
intelligent tutors
epistemic emotion
affective dynamics model

Access to Document

10.1007/978-3-031-36272-9_70Licence: All Rights Reserved

Affective Dynamic_RUAN_DOA03042023_AFV_CC BYAccepted author manuscript, 344 KBLicence: Creative Commons: Attribution (CC-BY)

https://link.springer.com/chapter/10.1007/978-3-031-36272-9_70#Abs1Licence: All Rights Reserved

Cite this

Ruan, X., Palansuriya, C., & Constantin, A. (2023). Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education. In Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings (Vol. 13916, pp. 774-779). (Lecture Notes in Computer Science; Vol. 13916). Springer. https://doi.org/10.1007/978-3-031-36272-9_70

Ruan, Xingran ; Palansuriya, Charaka ; Constantin, Aurora. / Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education. Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings. Vol. 13916 Springer, 2023. pp. 774-779 (Lecture Notes in Computer Science).

@inproceedings{1436b98b71474781badc00e640e5ddf1,

title = "Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education",

abstract = "Facial expressions of learners are relevant to their learning outcomes. The recognition of their emotional status influences the benefits of instruction or feedback provided by the intelligent tutor in education. However, learners{\textquoteright} emotions expressed during interactions with the intelligent tutor are mostly detected by self-reports of learners or judges who observe them in manually. The automated Facial Emotion Recognition (FER) task has been a challenging problem for intelligent tutors. The state-of-art automated FER methods target six basic emotions instead of learning-related emotions (e.g., neutral, confused, frustrated, and bored). Thus our research contributes to training a machine learning (ML) model to recognise learning-related emotions for intelligent tutors automatically, based on an Affective Dynamics (AD) model. We implement the AD model into our loss function (AD-loss) to fine tune the ML model. In the test scenario, the AD-loss method improves the performance of state-of-art FER algorithms.",

keywords = "facial emotion recognition, intelligent tutors, epistemic emotion, affective dynamics model",

author = "Xingran Ruan and Charaka Palansuriya and Aurora Constantin",

note = "Publisher Copyright: {\textcopyright} 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.; 24th International Conference on Artificial Intelligence in Education, AIED 2023 ; Conference date: 03-07-2023 Through 07-07-2023",

year = "2023",

month = jun,

day = "25",

doi = "10.1007/978-3-031-36272-9_70",

language = "English",

isbn = "9783031362712",

volume = "13916",

series = "Lecture Notes in Computer Science",

publisher = "Springer",

pages = "774--779",

booktitle = "Artificial Intelligence in Education",

url = "https://www.aied2023.org/",

}

Ruan, X, Palansuriya, C & Constantin, A 2023, Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education. in Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings. vol. 13916, Lecture Notes in Computer Science, vol. 13916, Springer, pp. 774-779, 24th International Conference on Artificial Intelligence in Education, Tokyo, Japan, 3/07/23. https://doi.org/10.1007/978-3-031-36272-9_70

Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education. / Ruan, Xingran; Palansuriya, Charaka ; Constantin, Aurora.
Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings. Vol. 13916 Springer, 2023. p. 774-779 (Lecture Notes in Computer Science; Vol. 13916).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education

AU - Ruan, Xingran

AU - Palansuriya, Charaka

AU - Constantin, Aurora

N1 - Conference code: 24

PY - 2023/6/25

Y1 - 2023/6/25

N2 - Facial expressions of learners are relevant to their learning outcomes. The recognition of their emotional status influences the benefits of instruction or feedback provided by the intelligent tutor in education. However, learners’ emotions expressed during interactions with the intelligent tutor are mostly detected by self-reports of learners or judges who observe them in manually. The automated Facial Emotion Recognition (FER) task has been a challenging problem for intelligent tutors. The state-of-art automated FER methods target six basic emotions instead of learning-related emotions (e.g., neutral, confused, frustrated, and bored). Thus our research contributes to training a machine learning (ML) model to recognise learning-related emotions for intelligent tutors automatically, based on an Affective Dynamics (AD) model. We implement the AD model into our loss function (AD-loss) to fine tune the ML model. In the test scenario, the AD-loss method improves the performance of state-of-art FER algorithms.

AB - Facial expressions of learners are relevant to their learning outcomes. The recognition of their emotional status influences the benefits of instruction or feedback provided by the intelligent tutor in education. However, learners’ emotions expressed during interactions with the intelligent tutor are mostly detected by self-reports of learners or judges who observe them in manually. The automated Facial Emotion Recognition (FER) task has been a challenging problem for intelligent tutors. The state-of-art automated FER methods target six basic emotions instead of learning-related emotions (e.g., neutral, confused, frustrated, and bored). Thus our research contributes to training a machine learning (ML) model to recognise learning-related emotions for intelligent tutors automatically, based on an Affective Dynamics (AD) model. We implement the AD model into our loss function (AD-loss) to fine tune the ML model. In the test scenario, the AD-loss method improves the performance of state-of-art FER algorithms.

KW - facial emotion recognition

KW - intelligent tutors

KW - epistemic emotion

KW - affective dynamics model

U2 - 10.1007/978-3-031-36272-9_70

DO - 10.1007/978-3-031-36272-9_70

M3 - Conference contribution

SN - 9783031362712

VL - 13916

T3 - Lecture Notes in Computer Science

SP - 774

EP - 779

BT - Artificial Intelligence in Education

PB - Springer

T2 - 24th International Conference on Artificial Intelligence in Education

Y2 - 3 July 2023 through 7 July 2023

ER -

Ruan X, Palansuriya C , Constantin A. Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education. In Artificial Intelligence in Education: 24th International Conference, AIED 2023, Tokyo, Japan, July 3–7, 2023, Proceedings. Vol. 13916. Springer. 2023. p. 774-779. (Lecture Notes in Computer Science). doi: 10.1007/978-3-031-36272-9_70

Affective Dynamic based Technique for Facial Emotion Recognition (FER) to Support Intelligent Tutors in Education

Abstract / Description of output

Publication series

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Fingerprint

Cite this