Multiagent Model-Based Credit Assignment for Continuous Control

Dongge Han, Chris Xiaoxuan Lu, Tomasz Michalak, Michael Wooldridge

Research output: Chapter in Book/Report/Conference proceedingConference contribution


Deep reinforcement learning (RL) has recently shown great promise in robotic continuous control tasks. Nevertheless, prior research in this vein center around the centralized learning setting that largely relies on the communication availability among all the components of a robot. However, agents in the real world often operate in a decentralised fashion without communication due to latency requirements, limited power budgets and safety concerns. By formulating robotic components as a system of decentralised agents, this work presents a decentralised multiagent reinforcement learning framework for continuous control. To this end, we first develop a cooperative multiagent PPO framework that allows for centralized optimisation during training and decentralised operation during execution. However, the system only receives a global reward signal which is not attributed towards each agent. To address this challenge, we further propose a generic game-theoretic credit assignment framework which computes agent-specific reward signals. Last but not least, we also incorporate a model-based RL module into our credit assignment framework, which leads to significant improvement in sample efficiency. Finally, we empirically demonstrate the effectiveness of our framework on Mujoco locomotion control tasks.
Original languageEnglish
Title of host publicationProceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems
PublisherACM Association for Computing Machinery
Number of pages9
ISBN (Electronic)9781450392136
Publication statusPublished - 9 May 2022
Event21st International Conference on Autonomous Agents and Multiagent Systems - Auckland, New Zealand
Duration: 9 May 202213 May 2022

Publication series

NameAAMAS '22
PublisherInternational Foundation for Autonomous Agents and Multiagent Systems


Conference21st International Conference on Autonomous Agents and Multiagent Systems
Abbreviated titleAAMAS 2022
Country/TerritoryNew Zealand
Internet address


  • cooperative game theory
  • multiagent systems
  • locomotion
  • reinforcement learning


Dive into the research topics of 'Multiagent Model-Based Credit Assignment for Continuous Control'. Together they form a unique fingerprint.

Cite this