Natural Language Processing Approaches in Industrial Maintenance: : A Systematic Literature Review
Abstract
Industrial maintenance plays a crucial role in manufacturing by significantly reducing machine failure time and minimizing costs, especially in the revolution of Industry 4.0. Consequently, researchers and industrial engineers have continuously focused on this area. Manufacturing companies possess extensive maintenance reports or logs containing valuable textual information, which offers a new avenue for exploring effective industrial maintenance methods. Natural Language Processing (NLP), a subfield of Artificial Intelligence, has demonstrated remarkable potential in analyzing maintenance reports and achieving promising results in various tasks. This paper presents a comprehensive systematic literature review that specifically concentrates on the applications of NLP approaches employed in the field of industrial maintenance. Additionally, this review analyzed the datasets utilized in previous studies and the evaluation measures adopted, which can serve as a valuable resource for other researchers seeking potential solutions in maintenance. Furthermore, the paper discusses the challenges encountered in applying NLP to industrial maintenance and outlines future research directions in this domain. By conducting this systematic literature review, we provide a comprehensive understanding of the current state of NLP applications in industrial maintenance, identify gaps in the existing literature, and guide future research efforts in leveraging NLP techniques for enhanced maintenance practices.
References
[1]
M. Ghobakhloo, Industry 4.0, digitization, and opportunities for sustainability, Journal of cleaner production 252 (2020).
[2]
T.P. Carvalho, F.A. Soares, R. Vita, R.D.P. Francisco, J.P. Basto, S.G. Alcalá, A systematic literature review of machine learning methods applied to predictive maintenance, Computers & Industrial Engineering 137 (2019).
[3]
Z.M. Cinar, Q. Zeeshan, D. Solyali, O. Korhan, Simulation of Factory 4.0: A Review, in: Industrial Engineering in the Digital Disruption Era: Selected papers from the Global Joint Conference on Industrial Engineering and Its Application Areas, GJCIE 2019, Springer International Publishing, Gazimagusa, North Cyprus, Turkey, 2020, pp. 204–216. September 2-3, 2019.
[4]
Z.M. Cinar, A.A. Nuhu, Q. Zeeshan, O. Korhan, Digital twins for industry 4.0: a review, in: Industrial Engineering in the Digital Disruption Era: Selected papers from the Global Joint Conference on Industrial Engineering and Its Application Areas, GJCIE 2019, Springer International Publishing, Gazimagusa, North Cyprus, Turkey, 2020, pp. 193–203. September 2-3, 2019.
[5]
Çınar, Z. M., Abdussalam Nuhu, A., Zeeshan, Q., Korhan, O., Asmael, M., & Safaei, B. (2020). Machine learning in predictive maintenance towards sustainable smart manufacturing in industry 4.0. Sustainability, 12(19), 8211.
[6]
F. Tao, Q. Qi, A. Liu, A. Kusiak, Data-driven smart manufacturing, Journal of Manufacturing Systems 48 (2018) 157–169.
[7]
A. Diez-Olivan, J. Del Ser, D. Galar, B. Sierra, Data fusion and machine learning for industrial prognosis: Trends and perspectives towards Industry 4.0, Information Fusion 50 (2019) 92–111.
[8]
L. Xia, P. Zheng, X. Li, R.X. Gao, L. Wang, Toward cognitive predictive maintenance: A survey of graph-based approaches, Journal of Manufacturing Systems 64 (2022) 107–120.
[9]
G.A. Susto, A. Beghi, C. De Luca, A predictive maintenance system for epitaxy processes based on filtering and prediction techniques, IEEE Transactions on Semiconductor Manufacturing 25 (4) (2012) 638–649.
[10]
G.A. Susto, A. Schirru, S. Pampuri, S. McLoone, A. Beghi, Machine learning for predictive maintenance: A multiple classifier approach, IEEE transactions on industrial informatics 11 (3) (2014) 812–820.
[11]
J. Wan, S. Tang, D. Li, S. Wang, C. Liu, H. Abbas, A.V. Vasilakos, A manufacturing big data solution for active preventive maintenance, IEEE Transactions on Industrial Informatics 13 (4) (2017) 2039–2047.
[12]
D. Rajpathak, S. De, A data-and ontology-driven text mining-based construction of reliability model to analyze and predict component failures, Knowledge and Information Systems 46 (1) (2016) 87–113.
[13]
F. Wang, T. Xu, T. Tang, M. Zhou, H. Wang, Bilevel feature extraction-based text mining for fault diagnosis of railway systems, IEEE transactions on intelligent transportation systems 18 (1) (2016) 49–58.
[14]
J. Devlin, M.W. Chang, K. Lee, K. Toutanova, Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint (2018) arXiv:1810.04805.
[15]
Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R.R. Salakhutdinov, Q.V. Le, Xlnet: Generalized autoregressive pretraining for language understanding, Advances in neural information processing systems (2019) 32.
[16]
S.M.R. Naqvi, C. Varnier, J.M. Nicod, N. Zerhouni, M. Ghufran, Leveraging free-form text in maintenance logs through bert transfer learning, in: Progresses in Artificial Intelligence & Robotics: Algorithms & Applications: Proceedings of 3rd International Conference on Deep Learning, Artificial Intelligence and Robotics, (ICDLAIR) 2021, Springer International Publishing, Cham, 2022, pp. 63–75.
[17]
S.M.R. Naqvi, M. Ghufran, S. Meraghni, C. Varnier, J.M. Nicod, N. Zerhouni, Cbr-based decision support system for maintenance text using nlp for an aviation case study, in: 2022 Prognostics and Health Management Conference (PHM-2022 London), IEEE, 2022, pp. 344–349.
[18]
D. Khurana, A. Koli, K. Khatter, S. Singh, Natural language processing: State of the art, current trends and challenges, Multimedia tools and applications 82 (3) (2023) 3713–3744.
[19]
F. Ansari, Cost-based text understanding to improve maintenance knowledge intelligence in manufacturing enterprises, Computers & Industrial Engineering 141 (2020).
[20]
F. Ansari, L. Kohl, J. Giner, H. Meier, Text mining for AI enhanced failure detection and availability optimization in production systems, CIRP Annals 70 (1) (2021) 373–376.
[21]
G. Nota, A. Postiglione, R. Carvello, Text mining techniques for the management of predictive maintenance, Procedia Computer Science 200 (2022) 778–792.
[22]
S. Roberto, P. Fabiana, P. Giuditta, C. Sergio, NLP-based insights discovery for industrial asset and service improvement: an analysis of maintenance reports, IFAC-PapersOnLine 55 (2) (2022) 522–527.
[23]
H.M. Wallach, Topic modeling: beyond bag-of-words, in: Proceedings of the 23rd international conference on Machine learning, 2006, pp. 977–984.
[24]
Y. Yuan, R. Li, Y. Wang, T. Cao, J. Yang, Y. La, Application of the maintenance text data of transformers based on SimHash and Hamming distance algorithm, in: 2020 IEEE International Conference on High Voltage Engineering and Application (ICHVE), IEEE, 2020, pp. 1–4.
[25]
G.S. Manku, A. Jain, A. Das Sarma, Detecting near-duplicates for web crawling, in: Proceedings of the 16th international conference on World Wide Web, 2007, pp. 141–150.
[26]
J. Ramos, Using tf-idf to determine word relevance in document queries, Proceedings of the first instructional conference on machine learning 242 (1) (2003) 29–48.
[27]
Y. Zhang, R. Jin, Z.H. Zhou, Understanding bag-of-words model: a statistical framework, International journal of machine learning and cybernetics 1 (2010) 43–52.
[28]
Y. Goldberg, O. Levy, word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method, arXiv preprint (2014) arXiv:1402.3722.
[29]
A. Joulin, E. Grave, P. Bojanowski, T. Mikolov, Bag of tricks for efficient text classification, arXiv preprint (2016) arXiv:1607.01759.
[30]
P. Christopher Pereira, Text-mining maintenance records to automate the identification and grouping of failure modes, in: Offshore Technology Conference, OnePetro, 2020.
[31]
E. Öztürk, A. Solak, D. Bäcker, L. Weiss, K. Wegener, Analysis and relevance of service reports to extend predictive maintenance of large-scale plants, Procedia CIRP 107 (2022) 1551–1558.
[32]
N. Reimers, I. Gurevych, Sentence-bert: Sentence embeddings using siamese bert-networks, arXiv preprint (2019) arXiv:1908.10084.
[33]
A.S. Bhardwaj, A. Deep, D. Veeramani, S. Zhou, A custom word embedding model for clustering of maintenance records, IEEE Transactions on Industrial Informatics 18 (2) (2021) 816–826.
[34]
T. Sexton, M.P. Brundage, M. Hoffman, K.C. Morris, Hybrid datafication of maintenance logs from AI-assisted human tags, in: 2017 ieee international conference on big data (big data), IEEE, 2017, pp. 1769–1777.
[35]
C.C. Chang, C.J. Lin, LIBSVM: a library for support vector machines, ACM transactions on intelligent systems and technology (TIST) 2 (3) (2011) 1–27.
[36]
X. Gaozheng, F. Zixuan, W. Lifang, Research on SVM Fault Diagnosis Method Based on Text Feature Extraction Algorithm, in: 2020 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), IEEE, 2020, pp. 199–203.
[37]
T. Gunda, S. Hackett, L. Kraus, C. Downs, R. Jones, C. McNalley, A. Walker, A machine learning evaluation of maintenance records for common failure modes in PV inverters, IEEE Access 8 (2020) 211610–211620.
[38]
U. Rahman, M.U. Mahbub, Application of classification models on maintenance records through text mining approach in industrial environment, Journal of Quality in Maintenance Engineering 29 (1) (2023) 203–219.
[39]
G. Hamerly, C. Elkan, Learning the k in k-means, Advances in neural information processing systems (2003) 16.
[40]
Z. Yang, P. Baraldi, E. Zio, A novel method for maintenance record clustering and its application to a case study of maintenance optimization, Reliability Engineering & System Safety 203 (2020).
[41]
G. Li, Q. Zhang, R. Zheng, C. Wang, A fault analysis method based on text clustering, in: 2020 5th International Conference on Computer and Communication Systems (ICCCS), IEEE, 2020, pp. 93–98.
[42]
B.G. Marcot, T.D. Penman, Advances in Bayesian network modelling: Integration of modelling technologies, Environmental modelling & software 111 (2019) 386–393.
[43]
W. Chang, Z. Xu, M. You, S. Zhou, Y. Xiao, Y. Cheng, A Bayesian failure prediction network based on text sequence mining and clustering, Entropy 20 (12) (2018) 923.
[44]
T. Ademujimi, V. Prabhu, Fusion-Learning of Bayesian Network Models for Fault Diagnostics, Sensors 21 (22) (2021) 7633.
[45]
O.I. Abiodun, A. Jantan, A.E. Omolara, K.V. Dada, N.A. Mohamed, H. Arshad, State-of-the-art in artificial neural network applications: A survey, Heliyon 4 (11) (2018) e00938.
[46]
M. Navinchandran, M.E. Sharp, M.P. Brundage, T.B. Sexton, Discovering critical KPI factors from natural language in maintenance work orders, Journal of Intelligent Manufacturing (2021) 1–19.
[47]
Y. Zhang, Y. Wang, H. Gu, L. Liu, J. Zhang, H. Lin, Defect diagnosis method of main transformer based on operation and maintenance text mining, in: 2020 IEEE International Conference on High Voltage Engineering and Application (ICHVE), IEEE, 2020, pp. 1–4.
[48]
Z. Xu, B. Chen, S. Zhou, W. Chang, X. Ji, C. Wei, W. Hou, A text-driven aircraft fault diagnosis model based on a word2vec and priori-knowledge convolutional neural network, Aerospace 8 (4) (2021) 112.
[49]
K.M. Tarwani, S. Edem, Survey on recurrent neural network in natural language processing, Int. J. Eng. Trends Technol 48 (6) (2017) 301–304.
[50]
A. Graves, A. Graves, Long short-term memory, Supervised sequence labelling with recurrent neural networks (2012) 37–45.
[51]
J. Chung, C. Gulcehre, K. Cho, Y. Bengio, Empirical evaluation of gated recurrent neural networks on sequence modeling, arXiv preprint (2014) arXiv:1412.3555.
[52]
W. Yin, K. Kann, M. Yu, H. Schütze, Comparative study of CNN and RNN for natural language processing, arXiv preprint (2017) arXiv:1702.01923.
[53]
S.M.R. Naqvi, M. Ghufran, S. Meraghni, C. Varnier, J.M. Nicod, N. Zerhouni, Human knowledge centered maintenance decision support in digital twin environment, Journal of Manufacturing Systems 65 (2022) 528–537.
[54]
J.P. Usuga Cadavid, B. Grabot, S. Lamouri, R. Pellerin, A. Fortin, Valuing free-form text data from maintenance logs through transfer learning with camembert, Enterprise Information Systems 16 (6) (2022).
[55]
J.P. Usuga-Cadavid, S. Lamouri, B. Grabot, A. Fortin, Using deep learning to value free-form text data for predictive maintenance, International Journal of Production Research 60 (14) (2022) 4548–4575.
[56]
A. Blanco-M, P. Marti-Puig, K. Gibert, J. Cusidó, J. Solé-Casals, A text-mining approach to assess the failure condition of wind turbines using maintenance service history, Energies 12 (10) (2019) 1982.
[57]
M.D. Dangut, Z. Skaf, I.K. Jennions, An integrated machine learning model for aircraft components rare failure prognostics with log-based dataset, ISA transactions 113 (2021) 127–139.
[58]
S. Zhou, C. Wei, P. Li, A. Liu, W. Chang, Y. Xiao, A Text-Driven Aircraft Fault Diagnosis Model Based on Word2vec and Stacking Ensemble Learning, Aerospace 8 (12) (2021) 357.
[59]
Y. Jin, G. Fu, L. Qian, H. Liu, H. Wang, Representation and Extraction of Diesel Engine Maintenance Knowledge Graph with Bidirectional Relations Based on BERT and the Bi-LSTM-CRF Model, in: 2021 IEEE International Conference on e-Business Engineering (ICEBE), IEEE, 2021, pp. 126–133.
[60]
Z. Huang, W. Xu, K. Yu, Bidirectional LSTM-CRF models for sequence tagging, arXiv preprint (2015) arXiv:1508.01991.
[61]
J. Bian, Z. Mao, Y. Liu, Y. Ma, W. Liu, B. Jiang, Construction and Reasoning Method of Fault Knowledge Graph with application of Engineering Machinery, in: 2021 China Automation Congress (CAC), IEEE, 2021, pp. 2577–2581.
[62]
J. Chatterjee, N. Dethlefs, Automated Question-Answering for Interactive Decision Support in Operations & Maintenance of Wind Turbines, IEEE Access 10 (2022) 84710–84737.
Recommendations
Apply Natural Language Processing-Chatbot on Industry 4.0
Social Computing and Social MediaAbstractNLP, or natural language processing, is an area of artificial intelligence that has been studied for more than 50 years and allows computers to comprehend human language. NLP interprets and makes sense of spoken or written natural language inputs ...
Using Job Characteristics to Inform Interface Design for Industrial Maintenance Procedures
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing SystemsMaintenance of industrial equipment is done by fitters, electricians and other maintainers. For safety and quality control, maintainers must follow procedures; historically these have been paper-based. Asset-owning organisations seek to transition ...
Natural Language Processing in Game Studies Research: An Overview
Natural language processing (NLP) is a field of computer science and linguistics devoted to creating computer systems that use human (natural) language as input and/or output. The authors propose that NLP can also be used for game studies research. In ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
Copyright © 2024.
Publisher
Elsevier Science Publishers B. V.
Netherlands
Publication History
Published: 02 July 2024
Author Tags
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 12 Dec 2024
Other Metrics
Citations
View Options
View options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in