The Extended Dawid-Skene Model: Fusing Information from Multiple Data Schemas

Michael Camilleri, Christopher K I Williams

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

While label fusion from multiple noisy annotations is a well understood concept in data wrangling (tackled for example by the Dawid-Skene (DS) model), we consider the extended problem of carrying out learning when the labels themselves are not consistently annotated with the same schema. We show that even if annotators use disparate, albeit related, label-sets, we can still draw inferences for the underlying full label-set. We propose the Inter-Schema AdapteR (ISAR) to translate the fully-specified label-set to the one used by each annotator, enabling learning under such heterogeneous schemas, without the need to re-annotate the data. We apply our method to a mouse behavioural dataset, achieving significant gains (compared with DS) in out-of-sample log-likelihood (−3.40 to −2.39) and F1-score (0.785 to 0.864).
Original languageEnglish
Title of host publicationMachine Learning and Knowledge Discovery in Databases
Subtitle of host publicationInternational Workshops of ECML PKDD 2019, Würzburg, Germany, September 16–20, 2019, Proceedings, Part I
EditorsPeggy Cellier, Kurt Driessens
PublisherSpringer
Pages121-136
Number of pages16
ISBN (Electronic)978-3-030-43823-4
ISBN (Print)978-3-030-43822-7
DOIs
Publication statusPublished - 28 Mar 2020
EventThe European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases 2019 - https://ecmlpkdd2019.org/, Würzburg, Germany
Duration: 16 Sept 201920 Sept 2019

Publication series

NameCommunications in Computer and Information Science (CCIS)
PublisherSpringer, Cham
Volume1167
ISSN (Print)1865-0929
ISSN (Electronic)1865-0937

Conference

ConferenceThe European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases 2019
Abbreviated titleECMLPKDD 2019
Country/TerritoryGermany
CityWürzburg
Period16/09/1920/09/19

Keywords / Materials (for Non-textual outputs)

  • Multi-schema learning
  • Crowdsourcing
  • Annotations
  • Behavioural characterisation
  • Probabilistic modelling
  • Data wrangling

Fingerprint

Dive into the research topics of 'The Extended Dawid-Skene Model: Fusing Information from Multiple Data Schemas'. Together they form a unique fingerprint.

Cite this