基于MacBERT-BiLSTM-CRF的反恐领域细粒度实体识别

为验证基于深度学习的命名实体识别框架在反恐领域的有效性,参照ACE 2005实体标注规范,制订了细粒度反恐实体标签体系,构建了反恐实体语料集Anti-Terr-Corpus;提出基于MacBERT-BiLSTM-CRF的实体识别模型,通过能减少预训练和微调阶段差异的MacBERT(masked language modeling as correction bidirectional encoder representations from transformers)预训练语言模型获得动态字向量表达,送入双向长短时记忆(bidirectional long short-term memory, BiLSTM)和条件随机场(conditional random field, CRF)进行上下文特征编码和解码得到最佳实体标签;替换框架中的预训练语言模型进行对比实验。实验表明该模型可以有效获取反恐新闻中的重要实体。对比BiLSTM-CRF模型,MacBERT的加入提高了24.5%的F_1值;保持编码-解码层为BiLSTM-CRF时,加入MacBERT比加入ALBERT(a lite BERT)提高了5.1%的F_1值。可见,深度学习有利于反恐领域实体识别,能够利用公开反恐新闻文本为后续反恐形势预判服务,同时有助于反恐领域信息提取、知识图谱构建等基础性任务。

Fine-grained Entity Recognition based on MacBERT-BiLSTM-CRF in Anti-Terrorism Field

In order to verify the effectiveness of the named entity recognition framework based on deep learning in anti-terrorism field, a fine-grained anti-terrorism entity label system was developed referring to the ACE 2005 entity labeling specification to construct the tailored corpus called Anti-Terr-Corpus. The entity recognition model based on MacBERT-BILSTM-CRF was proposed. The dynamic word vector expression obtained by MacBERT, a kind of pretrained language models, could reduce the difference between pre-training and fine-tuning stages and was sent to BILSTM and CRF for context feature encoding and decoding to obtain the optimal entity tags. Replaced the pretrained language model in the framework to carry out the comparative experiments. Experiments show that this model can effectively obtain important entities in anti-terrorism news. Compared with the BILSTM-CRF model, the addition of MacBERT increased the F-value by 24.5%. When the encode-decoding layer is kept as BILSTM-CRF, the F-value of MacBERT is increased by 5.1% compared with that of ALBERT. It is concluded that deep learning contributes to entity identification in anti-terrorism field. The open anti-terrorism news can be used to serve for subsequent anti-terrorism situation prediction, being helpful for basic tasks such as information extraction and knowledge map construction in anti-terrorism field.