Abstract
Ultrafast laser scribing provides a new microscale materials processing capability. Due to the processing speed and high-quality requirement in modern industrial applications, it is important to measure and monitor quality characteristics in real time during a scribing process. Although deep learning models have been successfully applied for quality monitoring of laser welding and laser based additive manufacturing, these models require a large sample for training and a time-consuming data labelling procedure for a new application such as the laser scribing process. This paper presents a study on image-based characterization of laser scribing quality using a deep transfer learning model for several quality characteristics such as debris, scribe width, and straightness of a scribe line. Images taken from the laser scribes on intrinsic Si wafers are examined. These images are labelled in a large and a small dataset, respectively. The large dataset includes 154 and small dataset includes 21 images. A novel transfer deep convolutional neural network (TDCNN) model is proposed to learn and assess scribe quality using the small dataset. The proposed TDCNN is able to overcome the data challenge by leveraging a convolutional neural network (CNN) model already trained for basic geometric features. Appropriate image processing techniques are provided to measure scribe width and line straightness as well as total scribe and debris area using classified images with 96 percent accuracy. Validating model’s performance based on the small data set, the model trained with the large dataset has a similar accuracy of 97 percent. The trained TDCNN model was also applied to a different scribing application. With 10 additional images to retrain the model, the model accuracy performs as well as the original model at 96 percent. Based on the proposed TDCNN classification of debris on a scribed image of straight lines, two algorithms are proposed to compute scribe width and straightness. The results show that all the three quality characteristics of debris, scribe width, and scribe straightness can be effectively measured based on a much smaller set of images than regular CNN models would require.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). SegNet : A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495. https://doi.org/10.1109/TPAMI.2016.2644615
Bauer, E., & Kohavi, R. (1999). Empirical comparison of voting classification algorithms: Bagging, boosting, and variants. Machine Learning, 36(1), 105–139. https://doi.org/10.1023/a:1007515423169
Bosch, M., Foster, K., Christie, G., Wang, S., Hager, G. D., & Brown, M. (2019). Semantic stereo for incidental satellite images. IEEE Winter Conference on Applications of Computer Vision (WACV), 2019, 1524–1532. https://doi.org/10.1109/WACV.2019.00167
Bradley, D., & Roth, G. (2007). Adaptive thresholding using the integral image. Journal of Graphics Tools, 12(2), 13–21. https://doi.org/10.1080/2151237X.2007.10129236
Chua, Z. Y., Ahn, I. H., & Moon, S. K. (2017). Process monitoring and inspection systems in metal additive manufacturing: Status and applications. International Journal of Precision Engineering and Manufacturing-Green Technology, 4(2), 235–245. https://doi.org/10.1007/s40684-017-0029-7
Delli, U., & Chang, S. (2018). Automated process monitoring in 3D printing using supervised machine learning. Procedia Manufacturing, 26, 865–870. https://doi.org/10.1016/j.promfg.2018.07.111
Ding, J., Hu, X.-H., & Gudivada, V. (2021). A machine learning based framework for verification and validation of massive scale image data. IEEE Transactions on Big Data, 7(2), 451–467. https://doi.org/10.1109/TBDATA.2017.2680460
Ferguson, M., Ak, R., Lee, Y.-T. T., & Law, K. H. (2018). Automatic localization of casting defects with convolutional neural networks. 2017 IEEE International Conference on Big Data (BIGDATA), December, 1726–1735. https://doi.org/10.1109/bigdata.2017.8258115
Fotovvati, B., Wayne, S. F., Lewis, G., & Asadi, E. (2018). A Review on melt-pool characteristics in laser welding of metals. Advances in Materials Science and Engineering, 2018, 1–18. https://doi.org/10.1155/2018/4920718
GD&T Straightness. (2014). Geometric dimensioning and tolerancing (GD&T). https://www.gdandtbasics.com/straightness/
Gonzalez-Val, C., Pallas, A., Panadeiro, V., & Rodriguez, A. (2020). A convolutional approach to quality monitoring for laser manufacturing. Journal of Intelligent Manufacturing, 31(3), 789–795. https://doi.org/10.1007/s10845-019-01495-8
Grasso, M., Demir, A. G., Previtali, B., & Colosimo, B. M. (2018). In situ monitoring of selective laser melting of zinc powder via infrared imaging of the process plume. Robotics and Computer-Integrated Manufacturing, 49, 229–239. https://doi.org/10.1016/j.rcim.2017.07.001
Grasso, M., Laguzza, V., Semeraro, Q., & Colosimo, B. M. (2017). In-Process monitoring of selective laser melting: spatial detection of defects via image data analysis. Journal of Manufacturing Science and Engineering. https://doi.org/10.1115/1.4034715
Imani, F., Chen, R., Diewald, E., Reutzel, E., & Yang, H. (2019). Deep learning of variant geometry in layerwise imaging profiles for additive manufacturing quality control. Journal of Manufacturing Science and Engineering. https://doi.org/10.1115/1.4044420
Imani, F., Gaikwad, A., Montazeri, M., Rao, P., Yang, H., & Reutzel, E. (2018a). Layerwise in-process quality monitoring in laser powder bed fusion. Additive Manufacturing Bio and Sustainable Manufacturing. https://doi.org/10.1115/MSEC2018a-6477
Imani, F., Gaikwad, A., Montazeri, M., Rao, P., Yang, H., & Reutzel, E. (2018b). Process mapping and in-process monitoring of porosity in laser powder bed fusion using layerwise optical imaging. Journal of Manufacturing Science and Engineering, Transactions of the ASME. https://doi.org/10.1115/1.4040615
Jais, I. K. M., Ismail, A. R., & Nisa, S. Q. (2019). Adam optimization algorithm for wide and deep neural network. Knowledge Engineering and Data Science, 2(1), 41.
Ku, S., Pieters, B. E., Haas, S., Bauer, A., Ye, Q., & Rau, U. (2013). Electrical characterization of P3 isolation lines patterned with a UV laser incident from the film side on thin-film silicon solar cells. Solar Energy Materials and Solar Cells, 108, 87–92. https://doi.org/10.1016/j.solmat.2012.09.017
Leitz, K.-H., Redlingshöfer, B., Reg, Y., Otto, A., & Schmidt, M. (2011). Metal ablation with short and ultrashort laser pulses. Physics Procedia, 12, 230–238. https://doi.org/10.1016/j.phpro.2011.03.128
Li, L., Wu, Y., & Ye, M. (2014). Multi-class image classification based on fast stochastic gradient. Informatica, 38(145), 153.
Li, X., Zhang, L., Du, B., Zhang, L., & Shi, Q. (2017). Iterative reweighting heterogeneous transfer learning framework for supervised remote sensing image classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 10(5), 2022–2035. https://doi.org/10.1109/JSTARS.2016.2646138
Mayr, A., Lutz, B., Weigelt, M., Glabel, T., Kibkalt, D., Masuch, M., Riedel, A., & Franke, J. (2018). Evaluation of machine learning for quality monitoring of laser welding using the example of the contacting of hairpin windings. 2018 8th International Electric Drives Production Conference (EDPC), 1–7. https://doi.org/10.1109/EDPC.2018.8658346
Mousavi, H. K., Nazari, M., Takac, M., & Motee, N. (2019). Multi-agent image classification via reinforcement learning. IEEE International Conference on Intelligent Robots and Systems. https://doi.org/10.1109/IROS40897.2019.8968129
NADCA. (2015). NADCA Product Specification Standarts for Die Casting. In North American Die Casting Association. http://www.caldiecast.com/docs/Zinc-and-ZA-Alloy-Data.pdf
Najjartabar-Bisheh, M., Chang, S. I., & Lei, S. (2021). A layer-by-layer quality monitoring framework for 3D printing. Computers & Industrial Engineering. https://doi.org/10.1016/j.cie.2021.107314
Noh, H., Hong, S., & Han, B. (2015a). Learning deconvolution network for semantic segmentation. Proceedings of the IEEE International Conference on Computer Vision, 2015a Inter, 1520–1528. https://doi.org/10.1109/ICCV.2015a.178
Noh, H., Hong, S., & Han, B. (2015b). Learning deconvolution network for semantic segmentation. IEEE International Conference on Computer Vision (ICCV), 2015, 1520–1528. https://doi.org/10.1109/ICCV.2015.178
Otsu, N. (1979). Threshold selection method from gray-level histograms. IEEE Transactions on Systems, Man, and Cybernetics. https://doi.org/10.1109/TSMC.1979.4310076
Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: convolutional networks for biomedical image segmentation. 234–241. https://doi.org/10.1007/978-3-319-24574-4_28
Roozbahani, H., Marttinen, P., & Salminen, A. (2018). Real-time monitoring of laser scribing process of cigs solar panels utilizing high-speed camera. IEEE Photonics Technology Letters, 30(20), 1741–1744. https://doi.org/10.1109/LPT.2018.2867274
Roozbahani, H., Salminen, A., & Manninen, M. (2017). Real-time online monitoring of nanosecond pulsed laser scribing process utilizing spectrometer. Journal of Laser Applications, 29(2), 022208. https://doi.org/10.2351/1.4983520
Roozbahani, Hamid, Salminen, A., & Manninen, M. (2019). Real-time monitoring of laser scribing process utilizing high-speed camera. 2310, 2310. https://doi.org/10.2351/1.5118570
Sejnowski, T. J. (2019). Neural Information Processing Systems. The Deep Learning Revolution. https://doi.org/10.7551/mitpress/11474.003.0014
Sharma, S., Ball, J. E., Tang, B., Carruth, D. W., Doude, M., & Islam, M. A. (2019). Semantic segmentation with transfer learning for off-road autonomous driving. Sensors (switzerland), 19(11), 1–21. https://doi.org/10.3390/s19112577
Shevchik, S. A., Le-Quang, T., Farahani, F. V., Faivre, N., Meylan, B., Zanoli, S., & Wasmer, K. (2019). Laser welding quality monitoring via graph support vector machine with data adaptive kernel. IEEE Access, 7, 93108–93122. https://doi.org/10.1109/ACCESS.2019.2927661
Shevchik, S., Le-Quang, T., Meylan, B., Farahani, F. V., Olbinado, M. P., Rack, A., Masinelli, G., Leinenbach, C., & Wasmer, K. (2020). Supervised deep learning for real-time quality monitoring of laser welding with X-ray radiographic guidance. Scientific Reports, 10(1), 3389. https://doi.org/10.1038/s41598-020-60294-x
Unnikrishnan, A., Sowmya, V., & Soman, K. P. (2019). Deep learning architectures for land cover classification using red and near-infrared satellite images. Multimedia Tools and Applications, 78(13), 18379–18394. https://doi.org/10.1007/s11042-019-7179-2
Uzkent, B., Sheehan, E., Meng, C., Tang, Z., Burke, M., Lobell, D., & Ermon, S. (2019). Learning to interpret satellite images using wikipedia. IJCAI International Joint Conference on Artificial Intelligence, 2019-Augus, 3620–3626. https://doi.org/10.24963/ijcai.2019/502
Wang, X., Yu, X., Berg, M. J., Chen, P., Lacroix, B., Fathpour, S., & Lei, S. (2021). Curved waveguides in silicon written by a shaped laser beam. Optics Express, 29(10), 14201. https://doi.org/10.1364/OE.419074
Wang, X., Yu, X., Berg, M., DePaola, B., Shi, H., Chen, P., Xue, L., Chang, X., & Lei, S. (2020). Nanosecond laser writing of straight and curved waveguides in silicon with shaped beams. Journal of Laser Applications, 32(2), 022002. https://doi.org/10.2351/1.5139973
Wang, X., Yu, X., Shi, H., Tian, X., Chambonneau, M., Grojo, D., DePaola, B., Berg, M., & Lei, S. (2019). Characterization and control of laser induced modification inside silicon. Journal of Laser Applications, 31(2), 022601. https://doi.org/10.2351/1.5096086
Xie, X., Ho, J. W. K., Murphy, C., Kaiser, G., Xu, B., & Chen, T. Y. (2011). Testing and validating machine learning classifiers by metamorphic testing. Journal of Systems and Software, 84(4), 544–558. https://doi.org/10.1016/j.jss.2010.11.920
Yao, Y. L., Chen, H., & Zhang, W. (2005). Time scale effects in laser material removal: A review. The International Journal of Advanced Manufacturing Technology, 26(5–6), 598–608. https://doi.org/10.1007/s00170-003-2026-y
Yuan, B., Giera, B., Guss, G., Matthews, I., & Mcmains, S. (2019). Semi-supervised convolutional neural networks for in-situ video monitoring of selective laser melting. IEEE Winter Conference on Applications of Computer Vision (WACV), 2019, 744–753. https://doi.org/10.1109/WACV.2019.00084
Zhang, B., Hong, K.-M., & Shin, Y. C. (2020). Deep-learning-based porosity monitoring of laser welding process. Manufacturing Letters, 23, 62–66. https://doi.org/10.1016/j.mfglet.2020.01.001
Zhao, X., Cao, Y., Nian, Q., Shin, Y. C., & Cheng, G. (2014). Precise selective scribing of thin-film solar cells by a picosecond laser. Applied Physics A, 116(2), 671–681. https://doi.org/10.1007/s00339-014-8330-6
Zoph, B., & Le, Q. V. (2019). Neural architecture search with reinforcement learning. 5th International Conference on Learning Representations, ICLR 2017 - Conference Track Proceedings, 1–16.
Acknowledgements
Financial support for this work by the National Science Foundation under grants CMMI-1903740 is gratefully acknowledged.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Bisheh, M.N., Wang, X., Chang, S.I. et al. Image-based characterization of laser scribing quality using transfer learning. J Intell Manuf 34, 2307–2319 (2023). https://doi.org/10.1007/s10845-022-01926-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10845-022-01926-z