The Shapley Value of Classifiers in Ensemble Games

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

What is the value of an individual model in an ensemble of binary classifiers? We answer this question by introducing a class of transferable utility cooperative games called ensemble games. In machine learning ensembles, pre-trained models cooperate to make classification decisions. To quantify the importance of models in these ensemble games, we define Troupe - an efficient algorithm that allocates payoffs based on approximate Shapley values of the classifiers. We argue that the Shapley value of models in these games is an effective decision metric for choosing a high-performing subset of models from the ensemble. Our analytical findings prove that our Shapley value estimation scheme is precise and scalable; its performance increases with the size of the dataset and ensemble. Empirical results on real-world graph classification tasks demonstrate that our algorithm produces high-quality estimates of the Shapley value. We find that Shapley values can be utilized for ensemble pruning and that adversarial models receive a low valuation. Complex classifiers are frequently found to be responsible for both correct and incorrect classification decisions.
Original languageEnglish
Title of host publicationCIKM '21: Proceedings of the 30th ACM International Conference on Information & Knowledge Management
PublisherAssociation for Computing Machinery (ACM)
Pages1558–1567
Number of pages10
ISBN (Electronic)9781450384469
DOIs
Publication statusPublished - 26 Oct 2021
Event30th ACM International Conference on Information and Knowledge Management - Online, Gold Coast, Australia
Duration: 1 Nov 20215 Nov 2021
https://www.cikm2021.org/

Conference

Conference30th ACM International Conference on Information and Knowledge Management
Abbreviated titleCIKM 2021
Country/TerritoryAustralia
CityGold Coast
Period1/11/215/11/21
Internet address

Keywords

  • game theory
  • ensembles
  • Shapley value
  • random forest

Fingerprint

Dive into the research topics of 'The Shapley Value of Classifiers in Ensemble Games'. Together they form a unique fingerprint.

Cite this