Not a cute stroke: Analysis of Rule- and Neural Network-based Information Extraction Systems for Brain Radiology Reports

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We present an in-depth comparison of three clinical information extraction (IE) systems designed to perform entity recognition and negation detection on brain imaging reports: EdIE-R, a bespoke rule-based system, and two neural network models, EdIE-BiLSTM and EdIE-BERT, both multi-task learning models with a BiLSTM and BERT encoder respectively. We compare our models both on an in-sample and an out-of-sample dataset containing mentions of stroke findings and draw on our error analysis to suggest improvements for effective annotation when building clinical NLP models for a new domain. Our analysis finds that our rule-based system outperforms the neural models on both datasets and seems to generalise to the out-of-sample dataset. On the other hand, the neural models do not generalise negation to the out-of-sample dataset, despite metrics on the in-sample dataset suggesting otherwise.
Original languageEnglish
Title of host publicationProceedings of the 11th International Workshop on Health Text Mining and Information Analysis
PublisherAssociation for Computational Linguistics
Pages24–37
Number of pages14
ISBN (Print)978-1-952148-81-1
Publication statusPublished - 20 Nov 2020
EventThe 11th International Workshop on Health Text Mining and Information Analysis at EMNLP 2020 - Online Workshop
Duration: 20 Nov 202020 Nov 2020

Workshop

WorkshopThe 11th International Workshop on Health Text Mining and Information Analysis at EMNLP 2020
Abbreviated titleLOUHI 2020
CityOnline Workshop
Period20/11/2020/11/20

Fingerprint Dive into the research topics of 'Not a cute stroke: Analysis of Rule- and Neural Network-based Information Extraction Systems for Brain Radiology Reports'. Together they form a unique fingerprint.

Cite this