More Web Proxy on the site http://driver.im/

research-article

A Prompt-based Few-shot Machine Reading Comprehension Model for Intelligent Bridge Management

Authors:

Qiao XiaoAuthors Info & Claims

EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering

Pages 946 - 950

https://doi.org/10.1145/3573428.3573599

Published: 15 March 2023 Publication History

Abstract

Bridge inspection reports are an important data source in the bridge management process, and they contain a large amount of fine-grained information. However, the research on machine reading comprehension (MRC) methods for this field is insufficient, and annotating large scale domain-specific corpus is time-consuming. This paper presented a novel prompt-based few-shot MRC approach for intelligent bridge management. The proposed model uses the pretrained model MacBERT as backbone. The prompt templates are designed based on some domain-specific heuristic rules. The experimental results show that our model outperforms the baseline models in different few-shot settings. The proposed model can provide technical support for the construction of automatic question answering system in the field of bridge management.

References

[1]

Farooq Henna and kaushik Baijnath, 2020. Review of Deep Learing Techniques for Improving the perfornace of Machine Reading Comprehension Problem. In Proceedings of the International Conference on Intelligent Computing and Control Systems, (2020), 928-935.

[2]

Shanshan Liu, Xin Zhang, Sheng Zhang, Hui Wang and Weiming Zhang, 2019. Neural Machine Reading Comprehnsion: Methods and Trends, Applied Sciences, 3698.

[3]

Seungwon Baek, WooyongJung, Seung H.Hana, 2021. A critical review of text-based research in construction: data source, analysis method, and implications, Automation in Construction, 132 (2021) 103915.

[4]

Ren Li; Tianjin Mo; Jianxi Yang; Shixin Jiang; Tong Li; Yiming Liu, 2021. Ontologies-based Domain Knowledge Modeling and Heterogeneous Sensor Data Integration for Bridge Health Monitoring Systems, IEEE Transactions on Industrial Informatics, 17(1):321-332.

[5]

Jianxi Yang, Fangyue Xiang, Ren Li, Luyi Zhang, Xiaoxia Yang, Shixin Jiang, Hongyi Zhang, Di Wang, Xinlong Liu, 2022. Intelligent bridge management via big data knowledge engineering, Automation in Construction, 135, 104118.

[6]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova, 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 4171-4186.

[7]

Yinhan Liu, Myle Ott, Naman Goyal, 2019. RoBERTa: a robustly optimized BERT pretraining approach, arXiv preprint.

[8]

Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang, Guoping Hu, 2020. Revisiting pre-trained models for Chinese natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2020, 657-668.

[9]

Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig, 2022. Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing, ACM Computing Surveys, (2022).

Digital Library

[10]

Mike Lewis, Yinhan Liu, Naman Goya, 2020. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics,7871-7880.

[11]

Yingjie Gu, Xiaolin Gui, Defu Li, Yi Shen, Dong Liao, 2020. Survey of machine reading comprehension based neural network, Journal of Software, 31(7), 2095-2126.

[12]

Hu Xu, Bing Liu, Lei Shu, Philip Yu, 2019. BERT Post-training for review reading comprehension and aspect-based sentiment analysis. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2324–2335.

[13]

Xu Han, Weilin Zhao, Ning Ding, Zhiyuan Liu, Maosong Sun, 2021. PTR: Prompt Tuning with Rules for Text Classification. 11259, 2021.

[14]

Hongbin Ye, Ningyu Zhang, Shumin Deng,et al., 2022. Ontology-enhanced Prompt-tuning for Few-shot Learning. In Proceedings of the ACM Web Conference 2022, 778-787.

Digital Library

[15]

Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Ziqing Yang, 2021. Pre-training with whole word masking for Chinese BERT, IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 3504-3514.

Digital Library

[16]

Ori Ram, Yuval Kirstain, Jonathan Berant, Amir Globerson, Omer Levy, 2021. Few-shot question answering by pretraining span selection. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, 3066-3079.

Cited By

Wang XLuo ZChen Y(2024)Automatic Generation of Critical Audit Matters (CAMs) Using LSTM–MacBert-Based Dual-Stream Transfer LearningIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.338549311:5(6435-6452)Online publication date: Oct-2024
https://doi.org/10.1109/TCSS.2024.3385493
XU YFAN YBAO YLI H(2024)Few-shot learning for structural health diagnosis of civil infrastructureAdvanced Engineering Informatics10.1016/j.aei.2024.10265062(102650)Online publication date: Oct-2024
https://doi.org/10.1016/j.aei.2024.102650

Recommendations

CorefPM: Coreference Resolution as Linguistic Prompt Prediction and Machine Reading Comprehension
CCRIS '22: Proceedings of the 2022 3rd International Conference on Control, Robotics and Intelligent System

Coreference resolution is a key challenge in natural language processing and is considered as a good test of machine intelligence. Recent coreference resolution prefers to supervised methods due to its superior performance. But supervised methods suffer ...
Multi-task joint training model for machine reading comprehension
Abstract
Machine reading comprehension is an important topic in natural language processing. However, the redundancy of text and the diversity of answer types are neglected in existing machine reading comprehension. To tackle these issues, a multi-task ...
CPCM-MRC: A Few-shot Machine Reading Comprehension Approach for Intelligent Bridge Management via Continual Pre-training and Copy Mechanism
ICAICE '23: Proceedings of the 4th International Conference on Artificial Intelligence and Computer Engineering

Nowadays, bridges, as important transportation infrastructure, have attracted increasing attention in their maintenance processes. Among them, bridge inspection reports serve as crucial data sources, containing a wealth of fine-grained information ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Other conferences

EITCE '22: Proceedings of the 2022 6th International Conference on Electronic Information Technology and Computer Engineering

October 2022

1999 pages

ISBN:9781450397148

DOI:10.1145/3573428

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 March 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

EITCE 2022

EITCE 2022: 2022 6th International Conference on Electronic Information Technology and Computer Engineering

October 21 - 23, 2022

Xiamen, China

Acceptance Rates

Overall Acceptance Rate 508 of 972 submissions, 52%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
62
Total Downloads

Downloads (Last 12 months)22
Downloads (Last 6 weeks)2

Reflects downloads up to 31 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang XLuo ZChen Y(2024)Automatic Generation of Critical Audit Matters (CAMs) Using LSTM–MacBert-Based Dual-Stream Transfer LearningIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.338549311:5(6435-6452)Online publication date: Oct-2024
https://doi.org/10.1109/TCSS.2024.3385493
XU YFAN YBAO YLI H(2024)Few-shot learning for structural health diagnosis of civil infrastructureAdvanced Engineering Informatics10.1016/j.aei.2024.10265062(102650)Online publication date: Oct-2024
https://doi.org/10.1016/j.aei.2024.102650

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents