Ontological attention ensembles for capturing semantic concepts in ICD code prediction from clinical text

Matus Falis, Maciej Pajak, Aneta Lisowska, Patrick Schrempf, Lucas Deckers, Shadia Mikhael, Sotirios Tsaftaris, Alison O'Neil

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract / Description of output

We present a semantically interpretable system for automated ICD coding of clinical text documents. Our contribution is an ontological attention mechanism which matches the structure of the ICD ontology, in which shared attention vectors are learned at each level of the hierarchy, and combined into label-dependent ensembles. Analysis of the attention heads shows that shared concepts are learned by the lowest common denominator node. This allows child nodes to focus on the differentiating concepts, leading to efficient learning and memory usage. Visualisation of the multi-level attention on the original text allows explanation of the code predictions according to the semantics of the ICD ontology. On the MIMIC-III dataset we achieve a 2.7% absolute (11% relative) improvement from 0.218 to 0.245 macro-F1 score compared to the previous state of the art across 3,912 codes. Finally, we analyse the labelling inconsistencies arising from different coding practices which limit performance on this task.
Original languageEnglish
Title of host publicationProceedings of the Tenth International Workshop on Health Text Mining and Information Analysis (LOUHI 2019)
Place of PublicationHong Kong
PublisherAssociation for Computational Linguistics
Number of pages10
Publication statusPublished - 1 Nov 2019


Dive into the research topics of 'Ontological attention ensembles for capturing semantic concepts in ICD code prediction from clinical text'. Together they form a unique fingerprint.

Cite this