Question Answering System for Drilling Safety Based on Tri-BiLSTM-CNN

doi:10.11885/j.issn.1674-5086.2020.05.12.06

Abstract

Abstract: The FAQ question answering system in a specific field usually has the following three problems:(1) how to effectively represent sentences semantically; (2) how to effectively match sentences semantically; (3) how to segment domain words. To solve the above three problems, a deep learning model based on Triplet BiLSTM-CNN is proposed. Firstly, the bidirectional long-term memory network and convolutional neural network are combined to construct the network model, which makes full use of the advantages of BiSLTM in processing the serialized data and the advantages of CNN in capturing local features. Then, the Triplet parallel structure is used to match sentences. Finally, character vector is used instead of word vector to avoid the influence of segmentation error on the model. The experimental results on real data sets in the field of drilling safety show that Triplet BiLSTM-CNN model can better vectorize sentence semantics and significantly improve the accuracy of sentence similarity calculation, and the effect is significantly better than that of CNN and LSTM. The model is applied to the FAQ question answering system in the field of drilling safety, which can effectively reduce the labor cost, and is of great significance and application value to improve the efficiency and quality of drilling work.

Key words: drilling safety, question answering system, bidirectional long short term memory, convolution neural network, question similarity computation

CLC Number:

TE319

WANG Bing, ZHENG Yamei, CHEN Maoke, GAO Lingyun. Question Answering System for Drilling Safety Based on Tri-BiLSTM-CNN[J]. Journal of Southwest Petroleum University(Science & Technology Edition), 2020, 42(6): 157-164.

0
/ / Recommend

Add to citation manager EndNote|Ris|BibTeX

URL: http://journal15.magtechjournal.com/Jwk_xnzk/EN/10.11885/j.issn.1674-5086.2020.05.12.06

http://journal15.magtechjournal.com/Jwk_xnzk/EN/Y2020/V42/I6/157

References

[1] 郭天翼,彭敏,伊穆兰,等. 自然语言处理领域中的自动问答研究进展[J]. 武汉大学学报(理学版),2019,65(5):417-426. doi:10.14188/j.1671-8836.2019.05.001 GUO Tianyi, PENG Min, YI Mulan, et al. Advances in question answering in natural language processing[J]. Journal of Wuhan University (Natural Science Edition), 2019, 65(5):417-426. doi:10.14188/j.1671-8836.2019.05.001
[2] WADHWA S, CHANDU K R, NYBERG E. Comparative analysis of neural QA models on SQuAD[C]. arXiv:W18-2610, 2018. doi:10.18653/v1/W18-2610
[3] SU D, XU Y, WINATA G I, et al. Generalizing question answering system with pretrained language model finetuning[C]. Proceedings of the 2nd Workshop on Machine Reading for Question Answering, 2019. doi:10.18653/v1/D19-5827
[4] 曹建文,万福成. 面向自动问答系统的问句相似度计算研究[J]. 重庆大学学报, 2019, 42(9):115-123. doi:10.11835/j.issn.1000-582X.2019.09.013 CAO Jianwen, WAN Fucheng. Question similarity computing method for automatic question answering system[J]. Journal of Chongqing University, 2019, 42(9):115-123. doi:10.11835/j.issn.1000-582X.2019.09.013
[5] WAN S, DRAS M, DALE R, et al. Using dependencybased features to take the ‘para-farce’ out of paraphrase[C]. Proceedings of the Australasian Language Technology Workshop, 2006:131-138.
[6] DAS D, SMITH N A. Paraphrase identification as probabilistic quasi-synchronous recognition[C]. Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, 2009:468-476. doi:10.3115/1687878.1687944
[7] FERNANDO S, STEVENSON M. A semantic similarity approach to paraphrase detection[C]. Proceedings of the 11th Annual Research Colloquium of the UK Special Interest Group for Computational Linguistics, 2008:45-52.
[8] GUO W, DIAB M. Modeling sentences in the latent space[C]. Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics:Long Papersvolume, 2012:864-872.
[9] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521:436-444. doi:10.1038/nature14539
[10] 冯兴杰,张乐,曾云泽. 基于多注意力CNN的问题相似度计算模型[J]. 计算机工程, 2009, 45(9):284-290. doi:10.1967-8/j.issn.1000-3428.0052098 FENG Xingjie, ZHANG Le, ZENG Yunze. Question similarity calculation model based on multi-attention CNN[J]. Computer Engineering, 2009, 45(9):284-290. doi:10.19678/j.issn.1000-3428.0052098
[11] 黄江平,姬东鸿. 基于卷积网络的句子语义相似性模型[J]. 华南理工大学学报(自然科学版), 2017, 45(3):68-75. doi:10.3969/j.issn.1000-565X.20-17.03.010 HUANG Jiangping, JI Donghong. Convolutional net-work-based semantic similarity model of sentences[J]. Journal of South China University of Technology (Natural Science Edition), 2017, 45(3):68-75. doi:10.3969/j.issn.1000-565X.20-17.03.010
[12] KALCHBRENNER N, GREFENSTETTE E, BLUNSOM P. A convolutional neural network for modelling sentences[C]. arXiv:P14-1062, 2014. doi:10.3115/v1/P14-1062
[13] YIN W, SCHÜTZE H. Convolutional neural network for paraphrase identification[C]. Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies, 2015:901-911.
[14] SOCHER R, HUANG E H, PENNIN J, et al. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection[C]. Advances in Neural Information Processing Systems, 2011:801-809.
[15] HU B, LU Z, LI H, et al. Convolutional neural network architectures for matching natural language sentences[C]. Advances in Neural Information Processing Systems, 2014:2042-2050.
[16] MUELLER J, THYAGARAJAN A. Siamese recurrent architectures for learning sentence similarity[C]. Thirtieth AAAI Conference on Artificial Intelligence, 2016.
[17] NECULOIU P, VERSTEEGH M, ROTARU M. Learning text similarity with siamese recurrent networks[C]. Proceedings of the 1st Workshop on Representation Learning for NLP, 2016:148-157.
[18] 徐雄. 基于深度学习的问答系统研究[J]. 湖北师范大学学报(自然科学版),2019,39(1):10-18. doi:10.3969/j.issn.2096-3149.2019.01.003 XU Xiong. Research on question answering system based on deep learning[J]. Journal of Hubei Normal University (Natural Science), 2019, 39(1):10-18. doi:10.3969/j.issn.2096-3149.2019.01.003
[19] 夏元昉. 基于深度学习的问答系统技术研究[D]. 杭州:浙江大学, 2017. XIA Yuanfang. Research on question answering system based on deep learning[D]. Hangzhou:Zhejiang University, 2017.
[20] 陈瑛,陈昂轩,董玉博,等. 基于LSTM的食品安全自动问答系统方法研究[J]. 农业机械学报, 2019, 50(S1):380-384. doi:10.6041/j.issn.1000-1298.2019.S0.058 CHEN Ying, CHEN Angxuan, DONG Yubo, et al. Methods of food safety question answering system based on LSTM[J]. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50(S1):380-384. doi:10.6041/j.issn.1000-1298.2019.S0.058
[21] 刘依红,杨波,孙宇宁,等. 基于BiLSTM的婚姻法自然语言问答[J]. 计算机工程与设计, 2019, 40(4):1190-1195. doi:10.16208/j.issn.1000-7024.2019.04.046 LIU Yihong, YANG Bo, SUN Yuling, et al. Q & A of natural language in marriage law based on BiLSTM[J]. Computer Engineering and Design, 2019, 40(4):1190-1195. doi:10.16208/j.issn1000-7024.2019.04.046
[22] HOFFER E, AILON N. Deep metric learning using triplet network[C]. International Workshop on Similarity-Based Pattern Recognition, Springer, Cham, 2015:84-92.
[23] JÜRGEN S. Deep learning in neural networks:An overview[J]. Neural Networks, 2015, 61:85-117. doi:10.1016/j.neunet.2014.09.003
[24] HOCHREITER S, SCHMIDHUBER J. Long short-term memory[J]. Neural Computation, 1997, 9(8):1735-1780. doi:10.1162/neco.1997.9.8.1735
[25] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[J]. Advances in Neural Information Processing Systems, 2013, 26:3111-3119.