Start Making Sense: Predicting confidence in virtual human interactions using biometric signals

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This project investigates the use of biometric data to predict confidence levels during task-focused interaction between humans and virtual humans. The project comprises of two main studies, the first of which examines the relationship between biometric signals – galvanic skin response (GSR), heart rate, facial expression and eye movements – and self-report levels of confidence during a task-oriented interaction between a human and a virtual human. Through the manipulation of the feedback and task demands, participants were exposed to unexpected situations and varying levels of ambiguity, resulting in a measurable range of perceived confidence as well as more implicit biometric and behavioural indicators of confidence and success. The second study utilises the paradigm and results from Experiment 1 to train an AI instruction giver to identify instances where behavioural and biometric feedback from a human signal low confidence, enabling it to modify or supplement its instructions accordingly. To ensure that the AI is acting in a useful way, and that the experimental manipulation and behavioural demonstrations of confidence are valid, the participant judges the perceived success of the interaction, as well as their trust in the AI under varying levels of feedback. This paradigm can then be adapted for use across a wide range of situations and scenarios; from interactions with virtual human avatars or agents via AR, VR, desktop or mobile devices, to fully embodied conversational agents or robots, this paradigm will enable a successful, smooth interactions between humans and AIs.
Original languageEnglish
Title of host publicationVolume 1 of the Proceedings of the joint meeting of the 12th International Conference on Measuring Behavior and the 6th Seminar on Behavioral Methods to be held in Krakow, Poland, October 15-18 2021
Publisherwww.measuringbehaviour.org
Pages72-75
Number of pages3
Volume1
ISBN (Print)978-90-74821-93-3
DOIs
Publication statusE-pub ahead of print - 7 Oct 2020
Event12th International Conference on Methods and Techniques in Behavioral Research and 6th International Seminar on Behavioral Methods - Krakow, Poland
Duration: 13 Oct 202115 Oct 2021
https://www.measuringbehavior.org/

Conference

Conference12th International Conference on Methods and Techniques in Behavioral Research and 6th International Seminar on Behavioral Methods
Abbreviated titleMeasuring Behaviour 2020-21
CountryPoland
CityKrakow
Period13/10/2115/10/21
Internet address

Fingerprint Dive into the research topics of 'Start Making Sense: Predicting confidence in virtual human interactions using biometric signals'. Together they form a unique fingerprint.

Cite this