A neural network multi-task learning approach to biomedical named entity recognition
Abstract Background Named Entity Recognition (NER) is a key task in biomedical text mining. Accurate NER systems require task-specific, manually-annotated datasets, which are expensive to develop and thus limited in size. Since such datasets contain related but different information, an interesting...
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMC
2017-08-01
|
Series: | BMC Bioinformatics |
Subjects: | |
Online Access: | http://link.springer.com/article/10.1186/s12859-017-1776-8 |
id |
doaj-662a634fad7d408780c368cb3b4f8e2d |
---|---|
record_format |
Article |
spelling |
doaj-662a634fad7d408780c368cb3b4f8e2d2020-11-24T21:25:19ZengBMCBMC Bioinformatics1471-21052017-08-0118111410.1186/s12859-017-1776-8A neural network multi-task learning approach to biomedical named entity recognitionGamal Crichton0Sampo Pyysalo1Billy Chiu2Anna Korhonen3Language Technology Laboratory, DTAL, University of CambridgeLanguage Technology Laboratory, DTAL, University of CambridgeLanguage Technology Laboratory, DTAL, University of CambridgeLanguage Technology Laboratory, DTAL, University of CambridgeAbstract Background Named Entity Recognition (NER) is a key task in biomedical text mining. Accurate NER systems require task-specific, manually-annotated datasets, which are expensive to develop and thus limited in size. Since such datasets contain related but different information, an interesting question is whether it might be possible to use them together to improve NER performance. To investigate this, we develop supervised, multi-task, convolutional neural network models and apply them to a large number of varied existing biomedical named entity datasets. Additionally, we investigated the effect of dataset size on performance in both single- and multi-task settings. Results We present a single-task model for NER, a Multi-output multi-task model and a Dependent multi-task model. We apply the three models to 15 biomedical datasets containing multiple named entities including Anatomy, Chemical, Disease, Gene/Protein and Species. Each dataset represent a task. The results from the single-task model and the multi-task models are then compared for evidence of benefits from Multi-task Learning. With the Multi-output multi-task model we observed an average F-score improvement of 0.8% when compared to the single-task model from an average baseline of 78.4%. Although there was a significant drop in performance on one dataset, performance improves significantly for five datasets by up to 6.3%. For the Dependent multi-task model we observed an average improvement of 0.4% when compared to the single-task model. There were no significant drops in performance on any dataset, and performance improves significantly for six datasets by up to 1.1%. The dataset size experiments found that as dataset size decreased, the multi-output model’s performance increased compared to the single-task model’s. Using 50, 25 and 10% of the training data resulted in an average drop of approximately 3.4, 8 and 16.7% respectively for the single-task model but approximately 0.2, 3.0 and 9.8% for the multi-task model. Conclusions Our results show that, on average, the multi-task models produced better NER results than the single-task models trained on a single NER dataset. We also found that Multi-task Learning is beneficial for small datasets. Across the various settings the improvements are significant, demonstrating the benefit of Multi-task Learning for this task.http://link.springer.com/article/10.1186/s12859-017-1776-8Multi-task learningConvolutional neural networksNamed entity recognitionBiomedical text mining |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Gamal Crichton Sampo Pyysalo Billy Chiu Anna Korhonen |
spellingShingle |
Gamal Crichton Sampo Pyysalo Billy Chiu Anna Korhonen A neural network multi-task learning approach to biomedical named entity recognition BMC Bioinformatics Multi-task learning Convolutional neural networks Named entity recognition Biomedical text mining |
author_facet |
Gamal Crichton Sampo Pyysalo Billy Chiu Anna Korhonen |
author_sort |
Gamal Crichton |
title |
A neural network multi-task learning approach to biomedical named entity recognition |
title_short |
A neural network multi-task learning approach to biomedical named entity recognition |
title_full |
A neural network multi-task learning approach to biomedical named entity recognition |
title_fullStr |
A neural network multi-task learning approach to biomedical named entity recognition |
title_full_unstemmed |
A neural network multi-task learning approach to biomedical named entity recognition |
title_sort |
neural network multi-task learning approach to biomedical named entity recognition |
publisher |
BMC |
series |
BMC Bioinformatics |
issn |
1471-2105 |
publishDate |
2017-08-01 |
description |
Abstract Background Named Entity Recognition (NER) is a key task in biomedical text mining. Accurate NER systems require task-specific, manually-annotated datasets, which are expensive to develop and thus limited in size. Since such datasets contain related but different information, an interesting question is whether it might be possible to use them together to improve NER performance. To investigate this, we develop supervised, multi-task, convolutional neural network models and apply them to a large number of varied existing biomedical named entity datasets. Additionally, we investigated the effect of dataset size on performance in both single- and multi-task settings. Results We present a single-task model for NER, a Multi-output multi-task model and a Dependent multi-task model. We apply the three models to 15 biomedical datasets containing multiple named entities including Anatomy, Chemical, Disease, Gene/Protein and Species. Each dataset represent a task. The results from the single-task model and the multi-task models are then compared for evidence of benefits from Multi-task Learning. With the Multi-output multi-task model we observed an average F-score improvement of 0.8% when compared to the single-task model from an average baseline of 78.4%. Although there was a significant drop in performance on one dataset, performance improves significantly for five datasets by up to 6.3%. For the Dependent multi-task model we observed an average improvement of 0.4% when compared to the single-task model. There were no significant drops in performance on any dataset, and performance improves significantly for six datasets by up to 1.1%. The dataset size experiments found that as dataset size decreased, the multi-output model’s performance increased compared to the single-task model’s. Using 50, 25 and 10% of the training data resulted in an average drop of approximately 3.4, 8 and 16.7% respectively for the single-task model but approximately 0.2, 3.0 and 9.8% for the multi-task model. Conclusions Our results show that, on average, the multi-task models produced better NER results than the single-task models trained on a single NER dataset. We also found that Multi-task Learning is beneficial for small datasets. Across the various settings the improvements are significant, demonstrating the benefit of Multi-task Learning for this task. |
topic |
Multi-task learning Convolutional neural networks Named entity recognition Biomedical text mining |
url |
http://link.springer.com/article/10.1186/s12859-017-1776-8 |
work_keys_str_mv |
AT gamalcrichton aneuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT sampopyysalo aneuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT billychiu aneuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT annakorhonen aneuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT gamalcrichton neuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT sampopyysalo neuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT billychiu neuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition AT annakorhonen neuralnetworkmultitasklearningapproachtobiomedicalnamedentityrecognition |
_version_ |
1725983525106089984 |