Improving RNN with Attention and Embedding for Adverse Drug Reactions

Chandra Pandey, Zina Ibrahim, Honghan Wu, Ehtesham Iqbal, Richard Dobson

Research output: Contribution to conferenceOtherpeer-review


Electronic Health Records (EHR) narratives are a rich source of information, embedding high-resolution information of value to secondary research use. However, because the EHRs are mostly in natural language free-text and highly ambiguity-ridden, many natural language processing algorithms have been devised around them to extract meaningful structured information about clinical entities. The performance of the algorithms however, largely varies depending on the training dataset as well as the effectiveness of the use of background knowledge to steer the learning process.

In this paper we study the impact of initializing the training of a neural network natural language processing algorithm with pre-defined clinical word embeddings to improve feature extraction and relationship classification between entities. We add our embedding framework to a bi-directional long short-term memory (Bi-LSTM) neural network, and further study the effect of using attention weights in neural networks for sequence labelling tasks to extract knowledge of Adverse Drug Reactions (ADRs). We incorporate unsupervised word embeddings using Word2Vec and GloVe from widely available medical resources such as Multiparameter Intelligent Monitoring in Intensive Care (MIMIC) II corpora, Unified Medical Language System (UMLS) as well as embed pharmaco lexicon from available EHRs. Our algorithm, implemented using two datasets, shows that our architecture outperforms baseline Bi-LSTM or Bi-LSTM networks using linear chain and Skip-Chain conditional random fields (CRF).
Original languageEnglish
Publication statusPublished - Jul 2017
EventDH '17: Proceedings of the 2017 International Conference on Digital Health -
Duration: 2 Jul 20175 Jul 2017


ConferenceDH '17: Proceedings of the 2017 International Conference on Digital Health

Fingerprint Dive into the research topics of 'Improving RNN with Attention and Embedding for Adverse Drug Reactions'. Together they form a unique fingerprint.

Cite this