Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics

Tiago Mota, Mohan Sridharan*, Aleš Leonardis

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract / Description of output

A robot’s ability to provide explanatory descriptions of its decisions and beliefs promotes effective collaboration with humans. Providing the desired transparency in decision making is challenging in integrated robot systems that include knowledge-based reasoning methods and data-driven learning methods. As a step towards addressing this challenge, our architecture combines the complementary strengths of non-monotonic logical reasoning with incomplete commonsense domain knowledge, deep learning, and inductive learning. During reasoning and learning, the architecture enables a robot to provide on-demand explanations of its decisions, the evolution of associated beliefs, and the outcomes of hypothetical actions, in the form of relational descriptions of relevant domain objects, attributes, and actions. The architecture’s capabilities are illustrated and evaluated in the context of scene understanding tasks and planning tasks performed using simulated images and images from a physical robot manipulating tabletop objects. Experimental results indicate the ability to reliably acquire and merge new information about the domain in the form of constraints, preconditions, and effects of actions, and to provide accurate explanations in the presence of noisy sensing and actuation.
Original languageEnglish
Article number242
Pages (from-to)1-18
Number of pages18
JournalSN Computer Science
Issue number4
Publication statusPublished - 1 Jul 2021

Keywords / Materials (for Non-textual outputs)

  • Explainable reasoning and learning
  • Non-monotonic logical reasoning
  • Deep learning
  • Scene understanding
  • Robotics


Dive into the research topics of 'Integrated Commonsense Reasoning and Deep Learning for Transparent Decision Making in Robotics'. Together they form a unique fingerprint.

Cite this