Explanation Styles for Trustworthy Autonomous Systems

David Robb; Xingkun Liu; Helen Hastie

doi:10.5555/3545946.3598913

Explanation Styles for Trustworthy Autonomous Systems

David Robb, Xingkun Liu, Helen Hastie

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Abstract

We present a study that explores the formulation of natural language explanations for managing the appropriate amount of trust in a remote autonomous system that fails to complete its mission. Online crowd-sourced participants were shown video vignettes of robots performing an inspection task. We measured participants' mental models, their confidence in their understanding of the robot behaviour and their trust in the robot. We found that including history in the explanation increases trust and confidence, and helps maintain an accurate mental model, but only if context is also included. In addition, our study exposes that some explanation formulations lacking in context can lead to misplaced participant confidence.

Original language	English
Title of host publication	Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)
Publisher	International Foundation for Autonomous Agents and Multiagent Systems
Pages	2298-2300
ISBN (Electronic)	9781450394321
DOIs	https://doi.org/10.5555/3545946.3598913
Publication status	Published - 30 May 2023
Event	The 22nd International Conference on Autonomous Agents and Multiagent Systems - ExCel London, London, United Kingdom Duration: 29 May 2023 → 2 Jun 2023 Conference number: 22 https://aamas2023.soton.ac.uk/

Conference

Conference	The 22nd International Conference on Autonomous Agents and Multiagent Systems
Abbreviated title	AAMAS 2023
Country/Territory	United Kingdom
City	London
Period	29/05/23 → 2/06/23
Internet address	https://aamas2023.soton.ac.uk/

Keywords / Materials (for Non-textual outputs)

Explanations
transparency
trust
robot faults
mental models

Access to Document

10.5555/3545946.3598913Licence: All Rights Reserved

https://aamas2023.soton.ac.uk/Licence: All Rights Reserved
https://dl.acm.org/doi/10.5555/3545946.3598913Licence: All Rights Reserved

Cite this

@inproceedings{87e394eb66904bc0851741ed2c8ac437,

title = "Explanation Styles for Trustworthy Autonomous Systems",

abstract = "We present a study that explores the formulation of natural language explanations for managing the appropriate amount of trust in a remote autonomous system that fails to complete its mission. Online crowd-sourced participants were shown video vignettes of robots performing an inspection task. We measured participants' mental models, their confidence in their understanding of the robot behaviour and their trust in the robot. We found that including history in the explanation increases trust and confidence, and helps maintain an accurate mental model, but only if context is also included. In addition, our study exposes that some explanation formulations lacking in context can lead to misplaced participant confidence.",

keywords = "Explanations, transparency, trust, robot faults, mental models",

author = "David Robb and Xingkun Liu and Helen Hastie",

note = "We thank our industrial partner, Total Energies, and our funder, UKRI, through grants: HUman-machine teaming for Maritime Environments (HUME) EP/V05676X/1, Trustworthy Autonomous Systems Node in Trust EP/V026682/1, and UK Robotics and Artificial Intelligence Hub for Offshore Energy Asset Integrity Management (The Orca Hub) EP/R026173/1.; The 22nd International Conference on Autonomous Agents and Multiagent Systems, AAMAS 2023 ; Conference date: 29-05-2023 Through 02-06-2023",

year = "2023",

month = may,

day = "30",

doi = "10.5555/3545946.3598913",

language = "English",

pages = "2298--2300",

booktitle = "Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)",

publisher = "International Foundation for Autonomous Agents and Multiagent Systems",

url = "https://aamas2023.soton.ac.uk/",

}

Robb, D, Liu, X & Hastie, H 2023, Explanation Styles for Trustworthy Autonomous Systems. in Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023). International Foundation for Autonomous Agents and Multiagent Systems, pp. 2298-2300, The 22nd International Conference on Autonomous Agents and Multiagent Systems, London, United Kingdom, 29/05/23. https://doi.org/10.5555/3545946.3598913

Explanation Styles for Trustworthy Autonomous Systems. / Robb, David; Liu, Xingkun; Hastie, Helen.
Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023). International Foundation for Autonomous Agents and Multiagent Systems, 2023. p. 2298-2300.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Explanation Styles for Trustworthy Autonomous Systems

AU - Robb, David

AU - Liu, Xingkun

AU - Hastie, Helen

N1 - Conference code: 22

PY - 2023/5/30

Y1 - 2023/5/30

N2 - We present a study that explores the formulation of natural language explanations for managing the appropriate amount of trust in a remote autonomous system that fails to complete its mission. Online crowd-sourced participants were shown video vignettes of robots performing an inspection task. We measured participants' mental models, their confidence in their understanding of the robot behaviour and their trust in the robot. We found that including history in the explanation increases trust and confidence, and helps maintain an accurate mental model, but only if context is also included. In addition, our study exposes that some explanation formulations lacking in context can lead to misplaced participant confidence.

AB - We present a study that explores the formulation of natural language explanations for managing the appropriate amount of trust in a remote autonomous system that fails to complete its mission. Online crowd-sourced participants were shown video vignettes of robots performing an inspection task. We measured participants' mental models, their confidence in their understanding of the robot behaviour and their trust in the robot. We found that including history in the explanation increases trust and confidence, and helps maintain an accurate mental model, but only if context is also included. In addition, our study exposes that some explanation formulations lacking in context can lead to misplaced participant confidence.

KW - Explanations

KW - transparency

KW - trust

KW - robot faults

KW - mental models

UR - https://pure.hw.ac.uk/ws/portalfiles/portal/91860345/Robb_AAMAS2023_Ext_Abs_cameraReady.pdf

U2 - 10.5555/3545946.3598913

DO - 10.5555/3545946.3598913

M3 - Conference contribution

SP - 2298

EP - 2300

BT - Proceedings of the 22nd International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2023)

PB - International Foundation for Autonomous Agents and Multiagent Systems

T2 - The 22nd International Conference on Autonomous Agents and Multiagent Systems

Y2 - 29 May 2023 through 2 June 2023

ER -

Explanation Styles for Trustworthy Autonomous Systems

Abstract

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

UK Robotics and Artificial Intelligence Hub for Offshore Energy Asset Integrity Management (ORCA)

Cite this

Explanation Styles for Trustworthy Autonomous Systems

Abstract

Conference

Keywords / Materials (for Non-textual outputs)

Access to Document

Other files and links

Fingerprint

Projects

UK Robotics and Artificial Intelligence Hub for Offshore Energy Asset Integrity Management (ORCA)

Cite this