Automatic for the people – and their personal case narratives

Technology / 10 May 2019

UMC data scientist Eva-Lisa Meldau presents the de-identification research at the Data Innovation Summit in Stockholm, March 2019. Photo: Data Innovation @datasweden2019

Deep learning shows promise removing personal identifiers from case reports, according to new work presented by UMC researchers.


Case narratives can be a valuable source of insight about adverse drug reactions, potentially adding details that are less readily available – or not available at all – from the more structured, standardised data fields of individual case safety reports. But sharing case narratives raises legal and ethical duties to protect the confidentiality of patient’s private details.

Currently, removing personal identifiers from case narratives is a manual task that can be both time consuming and tedious. However, new work from UMC suggests there may be a better way.

At the Data Innovation Summit 2019, held in Stockholm in March, UMC data scientist Eva-Lisa Meldau discussed the research into automatically de-identifying case narratives to protect patient privacy. 

The UMC researchers trained neural networks with more than 500 medical records, developing a deep learning algorithm that “read” the narratives in multiple ways to predict identifying data, assisted by standard natural language processing tools and manually constructed rules and dictionary queries.

“We developed the system to be conservative in a way that it only retains words in the text if it is highly confident that they are not personal identifiers,” said Meldau.

The results have been encouraging, suggesting that the algorithm could be trained to perform as well as or possibly better than a human annotator, albeit at the expense of removing more non-personal text.

The algorithm has so far only been developed using medical records from a de-identification challenge. “As we fine-tune the algorithm with annotated original narratives, we expect the performance to improve even further,” said Meldau.

“Automatic detection  and removal of personal identifiers in case  narratives using deep learning” is available at bit.ly/umc-posters.

“Automatic detection and removal of personal identifiers in case narratives using deep learning” is available at bit.ly/umc-posters.

Tags: ArtificialIntelligence, MachineLearning, privacy, DataScience

You may also like


Chinese researchers launch new drug safety platform

A group of Chinese scientists has launched the “Inquiring Drug Safety (iDS)” service, an online platform with a new approach to co-creating and sharing drug safety information.

Technology / 23 March 2020

WHODrug Global now available in Chinese

WHODrug Global Chinese meets increasing local and international regulatory requirements for the use of standardised drug data in clinical trials and drug safety monitoring.

Technology / 21 October 2019

Found in space – New method reveals related drug and reaction terms

Could natural language processing unlock powerful new methods in pharmacovigilance science? See how a journey through space could lead to a new dimension in case report analysis.

Technology / 15 April 2020

Our website uses cookies

Cookies are small text files held on your computer. They allow us to give you the best browsing experience possible and mean that we can understand how you use our site. Some cookies have already been set. You can delete and block cookies but parts of our site won't work without them. By using our website you accept our use of cookies.

Find out more