University of Limerick
Ji_2023_Uniting.pdf (2.84 MB)

Uniting multi-scale local feature awareness and the self-attention mechanism for named entity recognition

Download (2.84 MB)
journal contribution
posted on 2023-06-22, 08:18 authored by Lin ShiLin Shi, Xianming Zou, Chenxu Dai, Zhanlin Ji

In recent years, a huge amount of text information requires processing to support the diagnosis and treatment of diabetes in the medical field; therefore, the named entity recognition of diabetes (DNER) is giving rise to the popularity of this research topic within this particular field. Although the mainstream methods for Chinese medical named entity recognition can effectively capture global context information, they ignore the potential local information in sentences, and hence cannot extract the local context features through an efficient framework. To overcome these challenges, this paper constructs a diabetes corpus and proposes the RMBC (RoBERTa Multi-scale CNN BiGRU Self-attention CRF) model. This model is a named entity recognition model that unites multi-scale local feature awareness and the self-attention mechanism. This paper first utilizes RoBERTa-wwm to encode the characters; then, it designs a local context-wise module, which captures the context information containing locally important features by fusing multi-window attention with residual convolution at the multi-scale and adds a self-attention mechanism to address the restriction of the bidirectional gated recurrent unit (BiGRU) capturing long-distance dependencies and to obtain global semantic information. Finally, conditional random fields (CRF) are relied on to learn of the dependency between adjacent tags and to obtain the optimal tag sequence. The experimental results on our constructed private dataset, termed DNER, along with two benchmark datasets, demonstrate the effectiveness of the model in this paper. 



Mathematics 11(11), 2412



Other Funding information

National Key Research and Development Program of China

Department or School

  • Computer Science & Information Systems