Semantic Place Understanding for Human–Robot Coexistence—Toward Intelligent Workplaces

Stefano Rosa, Andrea Patanè, Chris Xiaoxuan Lu, Niki Trigoni

Research output: Contribution to journalArticlepeer-review


Recent introductions of robots to everyday scenarios have revealed unprecedented opportunities for collaboration and social interaction between robots and people. However, to date, such interactions are hampered by a significant challenge: having a semantic understanding of their environment. Even simple requirements, such as “a robot should always be in the kitchen when a person is there,” are difficult to implement without prior training. In this paper, we advocate that robot-people coexistence can be leveraged to enhance the semantic understanding of the shared environment and improve situation awareness. We propose a probabilistic framework that combines human activity sensor data generated by smart wearables with low-level localization data generated by robots. Based on this low-level information and leveraging colocation events between a user and a robot, it can reason about the two types of semantic information: first, semantic maps, i.e., the utility of each room and, second, space usage semantics, i.e., tracking humans and robots through rooms of different utilities. The proposed system relies on two-way sharing of information between the robot and the user. In the first phase, user activities indicative of room utility are inferred from wearable devices and shared with the robot, enabling it to gradually build a semantic map of the environment. In the second phase, via colocation events, the robot teaches the user device to recognize the type of room where they are colocated. Over time, robot and user become increasingly independent and capable of semantic scene understanding.
Original languageEnglish
Pages (from-to)160-170
Number of pages11
JournalIEEE Transactions on Human-Machine Systems
Issue number2
Early online date30 Oct 2018
Publication statusPublished - 1 Apr 2019


  • human-robot interaction
  • robot-people coexistence
  • human activity sensor data
  • low-level localization data
  • semantic information
  • space usage semantics
  • semantic map
  • semantic scene understanding
  • semantic place understanding
  • social interaction
  • human-robot coexistence
  • smart wearables
  • Semantics
  • Training
  • Activity recognition
  • Simultaneous localization and mapping
  • Hidden Markov models
  • human–computer interaction
  • intelligent robots


Dive into the research topics of 'Semantic Place Understanding for Human–Robot Coexistence—Toward Intelligent Workplaces'. Together they form a unique fingerprint.

Cite this