基于BERT的非招标采购实体关系抽取研究

doi:10.12267/j.issn.2096-5931.2023.06.001

摘要/Abstract

摘要：

提出了一种基于BERT的实体长度感知的实体关系抽取模型IL-BERT,该模型在经过BERT方法提取的实体特征向量上引入实体长度信息,以增强实体特征的表示能力。分别在公共数据集和非招标采购领域数据集上进行试验,结果表明,提出模型的数值均优于基线模型。

关键词: BERT, 实体关系抽取, 实体长度感知, 非招标采购

Abstract:

This paper proposes an entity relationship extraction model IL-BERT which is based on BERT and can be aware of the entity length. This model introduces entity length information into the entity feature vectors extracted with the BERT method to enhance the representation ability of entity features. Experiments are conducted on both public and non-bidding procurement datasets, and the results show that the F₁ values of the proposed model are better than those of the baseline model.

Key words: BERT, entity relationship extraction, entity length-aware model, non-bidding procurement

中图分类号:

F253.2

张朝阳. 基于BERT的非招标采购实体关系抽取研究[J]. 信息通信技术与政策, 2023, 49(6): 2-9.

ZHANG Zhaoyang. Research on entity relationship extraction for non-bidding procurement based on BERT[J]. Information and Communications Technology and Policy, 2023, 49(6): 2-9.

导出引用管理器 EndNote|Ris|BibTeX

链接本文:

http://ictp.caict.ac.cn/CN/10.12267/j.issn.2096-5931.2023.06.001

http://ictp.caict.ac.cn/CN/Y2023/V49/I6/2

图/表 11

参考文献 18

[1]	HENDRICKX I, KIM S N, KOZAREVA Z, et al. Semeval-2010 task 8: multi-way classification of semantic relations between pairs of nominals[C]//Proceedings of the NAACL HLT Workshop on Semantic Evaluations: Recent Achievements and Future Directions. ACL, 2009: 94-99.
[2]	SOCHER R, HUVAL B, MANNING C D, et al. Semantic compositionality through recursive matrix-vector spaces[C]//Proceedings of the 2012 joint conference on empirical methods in natural language processing and computational natural language learning. ACL, 2012: 1201-1211.
[3]	ZENG D, LIU K, LAI S, et al. Relation classifica tion via convolutional deep neural network[C]//Proceedings of the 25th International Conference on Computational Linguistics. Pennsylvania: ACL, 2014:2335-2344.
[4]	YU M, GORMLEY M, Dredze M. Factor-based compositional embedding models[C]//Conference and Workshop on Neural Information Processing Systems Workshop on Learning Semantics. Montreal: NIPS, 2014: 95-101.
[5]	SANTOS C N, XIANG B, ZHOU B. Classifying relations by ranking with convolutional neural networks[J]. Computer Science, 2015, 86(86):132-137.
[6]	SHEN Y, HUANG X J. Attention-based convolutional neural network for semantic relation extraction[C]// Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics:Technical Papers. Osaka: ICCL, 2016: 2526-2536.
[7]	LEE J, SEO S, CHOI Y S. Semantic relation classification via bidirectional LSTM networks with entity-aware attention using latent entity typing[J]. Symmetry, 2019, 11(6): 785-796. doi: 10.3390/sym11060785 URL
[8]	LI Q, JI H. Incremental Joint Extraction of Entity Mentions and Relations[C]//Proceedings of Meeting of the Association for Computational Linguistics. ACL, 2014: 402-412.
[9]	BAI T, GUAN H, WANG S, et al. Traditional Chinese medicine entity relation extraction based on CNN with segment attention[J]. Neural Computing and Applications, 2022: 1-10.
[10]	MIWA M, BANSAL M. End-to-end relation extraction using LSTMs onsequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Berlin:ACL, 2016: 1105-1116.
[11]	ZHANG Y, ZHONG V, CHEN D, et al. Position-aware attention and supervised data improve slot filling[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen: ACL, 2017:35-45.
[12]	DAI A M, LE Q V. Semi-supervised sequence learning[C]//International Conference on Neural nformation Processing Svstems. MiT Press, 2015: 3079-3087.
[13]	PETERS M E, AMMAR W, BHAGAVATULA C, et al. Semi-supervised sequence tagging with bidirectional language models[C]// Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. ACL, 2017:1756-1765.
[14]	HOWARD J, RUDER S. Universal language model fine-tuning for text classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. ACL, 2018:328-339.
[15]	ZHANG Y, QI P, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Brussels: ACL, 2018: 2205-2215.
[16]	DEVLIN J, CHANG M, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]// Procedings of the 2019 Conference of the North American Chapter of the Association for Computational Linquistics:Human Lanquage Tecinologies. Stroudsburg: ACL, 2019: 4171-4186.
[17]	WU S, HE Y. Enriching pre-trained language model with entity information for relation classification[C]// Proceedings of the 28th ACM international conference on information and knowledge management. ACM, 2019: 2361-2364.
[18]	GAO S, DU J, ZHANG X. Research on relation extraction method of Chinese electronic medical records based on BERT[C]//Proceedings of the 2020 6th international conference on computing and artificial intelligence. ACM, 2020: 487-490.

编辑推荐

Metrics

阅读次数

全文

2767

HTML			PDF

最新录用	在线预览	正式出版	最新录用	在线预览	正式出版
0	0	2549	0	0	218

来源	本网站	其他网站

次数	2726	41
比例	99%	1%

摘要

361

最新录用	在线预览	正式出版

0	0	361

来源	本网站	其他网站

次数	360	1
比例	100%	0%

数量(个)	采购评审领域数据集	Semeval 2010 Task
训练集样本数量	75 587	8 000
测试集样本数量	18 896	2 000
关系类型数量	5	18

数量(个)	采购评审领域数据集	Semeval 2010 Task
训练集样本数量	75 587	8 000
测试集样本数量	18 896	2 000
关系类型数量	5	18

参数名称	数值
批量大小(个/批)	16
最大长度(单词数/句)	128
学习率(%)	0.002
迭代轮次(轮)	5
丢弃率(%)	10

参数名称	数值
批量大小(个/批)	16
最大长度(单词数/句)	128
学习率(%)	0.002
迭代轮次(轮)	5
丢弃率(%)	10

模型	F₁值(%)
MVRNN	82.4
CNN+softmax	82.7
FCM	83.0
CR-CNN	84.1
AttentionCNN	85.9
Att-Pooling-CNN	88.0
Entity Attention Bi-LSTM	85.2
C-GCN	84.8
R-BERT	88.0
IL-BERT	88.3