[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Domain adaptive segmentation method for mechanical assembly based on iterative loops

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

During the assembly process of mechanical products, employing deep learning techniques for the semantic segmentation of assembly images enables real-time monitoring of irregularities, including incorrect or missing assemblies. However, most of the current monitoring methods based on deep learning adopt supervised learning. This requires a large number of labels according to different assembly specifications, which is time-consuming and laborious. To address this issue, this study designed a two-stage adaptive segmentation framework based on iterative loops for synthesis-physical assembly images, i.e., ILDA-Net (iterative loops domain adaptation network), which does not require any labeling of physical assemblies. In the adversarial learning stage, a trainable line-guided filter module and a line discriminator module are introduced for maintaining line features. The two modules are iteratively trained in a loop to continuously optimize the segmentation model. In the self-training stage, the edge segmentation quality is guaranteed by optimizing the segmentation model through utilizing unreliable pseudo-labels. Finally, this study constructed a set of semantic segmentation datasets for domain adaptation of synthetic-physical assembly images and conducted experiments on these datasets. Based on these experiments, the Dice coefficient can reach up to 89.33%, which demonstrating that the proposed method can be utilized for the physical assembly image segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets generated or analyzed during this study are available from the corresponding author on reasonable request.

References

  1. Wang J, Chen C, Dai C (2023) A mechanical assembly monitoring method based on domain adaptive semantic segmentation. Int J Adv Manuf Technol 128(1–2):625–637. https://doi.org/10.1007/s00170-023-11878-0

    Article  MATH  Google Scholar 

  2. Zamora-Hernández MA, Castro-Vargas JA, Azorin-Lopez J, Garcia-Rodriguez J (2021) Deep learning-based visual control assistant for assembly in industry 4.0. Comput Ind 131:103485. https://doi.org/10.1016/j.compind.2021.103485

    Article  MATH  Google Scholar 

  3. Wang KJ, Yan YJ (2021) A smart operator assistance system using deep learning for angle measurement. IEEE Trans Instrum Meas 70(5019104):1–14. https://doi.org/10.1109/TIM.2021.3124044

    Article  MATH  Google Scholar 

  4. Chen C, Zhang C, Wang J, Li D, Li Y, Hong J (2023) Semantic segmentation of mechanical assembly using selective kernel convolution UNet with fully connected conditional random field. Measurement 209:112499. https://doi.org/10.1016/j.measurement.2023.112499

    Article  MATH  Google Scholar 

  5. Yin XY, Fan XM, Zhu WM, Liu R (2019) Synchronous AR Assembly assistance and monitoring system based on ego-centric vision. Assembly Autom 39(1):1–16. https://doi.org/10.1108/AA-03-2017-032

    Article  MATH  Google Scholar 

  6. Deng ZW, Kong Q, Akira N, Yoshinaga T (2022) Hierarchical contrastive adaptation for cross-domain object detection. Mach Vis Appl 33(4):62. https://doi.org/10.1007/s00138-022-01317-7

    Article  MATH  Google Scholar 

  7. Zhang D, Ye M, Liu YG, Xiong L, Zhou LH (2022) Multi-source unsupervised domain adaptation for object detection. Inform Fusion 78:138–148. https://doi.org/10.1016/j.inffus.2021.09.011

    Article  MATH  Google Scholar 

  8. Saxena S, Pandey S, Khanna P (2018) A semi-supervised domain adaptation assembling approach for image classification. Pattern Anal Appl 21(3):813–827. https://doi.org/10.1007/s10044-017-0664-1

    Article  MathSciNet  MATH  Google Scholar 

  9. Yin YM, Yang Z, Hu HF, Wu XF (2022) Universal multi-source domain adaptation for image classification. Pattern Recogn 121:108238. https://doi.org/10.1016/j.patcog.2021.108238

    Article  MATH  Google Scholar 

  10. Luo X, Chen W, Liang ZF, Yang LQ, Wang SW, Li C (2024) Crots: cross-domain Teacher-Student Learning for source-free domain adaptive semantic segmentation. Int J Comput Vision 132(1):20–39. https://doi.org/10.1007/s11263-023-01863-1

    Article  Google Scholar 

  11. Tian YJ, Zhu SY (2022) Partial domain adaptation on semantic segmentation. IEEE Trans Circuits Syst Video Technol 32(6):3798–3809. https://doi.org/10.1109/TCSVT.2021.3116210

    Article  MATH  Google Scholar 

  12. Hoffman J et al (2018) Cycada: cycle-consistent adversarial domain adaptation. International conference on machine learning 80:1989–1998. https://doi.org/10.48550/arXiv.1711.03213

  13. Vesal S, Gu M, Kosti R, Maier A, Ravikumar N (2021) Adapt everywhere: unsupervised adaptation of point-clouds and entropy minimization for multi-modal cardiac image segmentation. IEEE Trans Med Imaging 40(7):1838–1851. https://doi.org/10.1109/TMI.2021.3066683

    Article  Google Scholar 

  14. Chen CL, Wang G (2021) IOSUDA: an unsupervised domain adaptation with input and output space alignment for joint optic disc and cup segmentation. Appl Intell 51(6):3880–3898. https://doi.org/10.1007/s10489-020-01956-1

    Article  MATH  Google Scholar 

  15. Hu S, Bonardi F, Bouchafa S, Sidibé D (2023) Multi-modal unsupervised domain adaptation for semantic image segmentation. Pattern Recogn 137:109299. https://doi.org/10.1016/j.patcog.2022.109299

    Article  MATH  Google Scholar 

  16. Ma S, Song K, Niu M, Tian H, Wang Y, Yan Y (2023) Shape consistent one-shot unsupervised domain adaptation for rail surface defect segmentation. Ieee Trans Industrial Inform 19(9):9667–9679. https://doi.org/10.1109/TII.2022.3233654

    Article  MATH  Google Scholar 

  17. Liu W et al (2021) Adversarial unsupervised domain adaptation for 3D semantic segmentation with multi-modal learning. ISPRS J Photogrammetry Remote Sens 176:211–221. https://doi.org/10.1016/j.isprsjprs.2021.04.012

    Article  MATH  Google Scholar 

  18. Huang J, Guan D, Xiao A, Lu S (2022) Multi-level adversarial network for domain adaptive semantic segmentation. Pattern Recogn 123:108384. https://doi.org/10.1016/j.patcog.2021.108384

    Article  MATH  Google Scholar 

  19. Zhao YY, Zhong Z, Luo ZM, Lee GH, Sebe N (2022) Source-free open compound domain adaptation in semantic segmentation. IEEE Trans Circuits Syst Video Technol 32(10):7019–7032. https://doi.org/10.1109/TCSVT.2022.3179021

    Article  MATH  Google Scholar 

  20. Zhao SC, Li B, Xu PF, Yue XY, Ding GG, Keutzer K (2021) MADAN: multi-source adversarial domain Aggregation Network for Domain Adaptation. Int J Comput Vision 129(8):2399–2424. https://doi.org/10.1007/s11263-021-01479-3

    Article  MATH  Google Scholar 

  21. Cao W et al (2023) A two-stage domain alignment method for multi-source domain fault diagnosis. Measurement 214:112818. https://doi.org/10.1016/j.measurement.2023.112818

    Article  MATH  Google Scholar 

  22. Zou Y, Yu Z, Kumar B, Wang J (2018) Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. Proc Eur Conf Comput Vis 11207:289–305. https://doi.org/10.1007/978-3-030-01219-9_18

    Article  MATH  Google Scholar 

  23. Hoyer L, Dai DX, Wang Q, Chen YH, Van Gool L (2023) Improving semi-supervised and domain-adaptive semantic segmentation with self-supervised depth estimation. Int J Comput Vision 131(8):2070–2096. https://doi.org/10.1007/s11263-023-01799-6

    Article  MATH  Google Scholar 

  24. Xie BH, Li S, Li MJ, Liu CH, Huang G, Wang GR (2023) SePiCo: semantic-guided pixel contrast for Domain Adaptive Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell 45(7):9004–9021. https://doi.org/10.1109/TPAMI.2023.3237740

    Article  Google Scholar 

  25. Yang J, An W, Wang S, Zhu X, Yan C, Huang J (2020) Label-driven reconstruction for domain adaptation in semantic segmentation. Proc Eur Conf Comput Vis 12372:480–498. https://doi.org/10.1007/978-3-030-58583-9_29

    Article  MATH  Google Scholar 

  26. Cheng YT, Wei FY, Bao JM, Chen D, Zhang WQ (2023) ADPL: adaptive dual path learning for Domain Adaptation of Semantic Segmentation. IEEE Trans Pattern Anal Mach Intell 45(8):9339–9356. https://doi.org/10.1109/TPAMI.2023.3248294

    Article  MATH  Google Scholar 

  27. Zhang LF, Lan M, Zhang J, Tao DC (2022) Stagewise unsupervised domain adaptation with adversarial self-training for Road Segmentation of Remote-sensing images. IEEE Trans Geosci Remote Sens 60:1–13. https://doi.org/10.1109/TGRS.2021.3104032

    Article  MATH  Google Scholar 

  28. Hong J, Yu S, Chen W (2022) Unsupervised domain adaptation for cross-modality liver segmentation via joint adversarial learning and self-learning. Appl Soft Comput 121:108729. https://doi.org/10.1016/j.asoc.2022.108729

    Article  Google Scholar 

  29. Zhu JY, Park T, Isola P, Efros AA (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks. Proceedings of international conference on computer vision pp 2242–2251. https://doi.org/10.1109/ICCV.2017.244

  30. Abbaszadeh Shahri A, Chunling S, Larsson S (2023) A hybrid ensemble-based automated deep learning approach to generate 3D geo-models and uncertainty analysis. Engineering with Computers 1–16:1. https://doi.org/10.1007/s00366-023-01852-5

    Article  MATH  Google Scholar 

  31. Abdar M et al (2021) A review of uncertainty quantification in deep learning: techniques, applications and challenges. Inform Fusion 76:243–297. https://doi.org/10.1016/j.inffus.2021.05.008

    Article  MATH  Google Scholar 

  32. Patwary MJ, Wang XZ (2019) Sensitivity analysis on initial classifier accuracy in fuzziness based semi-supervised learning. Inf Sci 490:93–112. https://doi.org/10.1016/j.ins.2019.03.036

    Article  MATH  Google Scholar 

  33. Asheghi R, Hosseini SA, Saneie M, Abbaszadeh Shahri A (2020) Updating the neural network sediment load models using different sensitivity analysis methods: a regional application. J Hydroinformatics 22(3):562–577. https://doi.org/10.2166/hydro.2020.098

    Article  Google Scholar 

  34. Naik DL, Kiran R (2021) A novel sensitivity-based method for feature selection. J Big Data 8(1):128. https://doi.org/10.1186/s40537-021-00515-w

    Article  MATH  Google Scholar 

  35. Krishna DS et al (2018) DRUNET: a dilated-residual U-Net deep learning network to segment optic nerve head tissues in optical coherence tomography images. Biomedical Opt Express 9(7):3244–3265. https://doi.org/10.1364/BOE.9.003244

    Article  MATH  Google Scholar 

  36. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Proc Med Image Comput Computer-Assisted Intervention 9351:234–241. https://doi.org/10.1007/978-3-319-24574-4_28

    Article  MATH  Google Scholar 

  37. Wang J, Zheng Z, Ma A, Lu X, Zhong Y (2021) LoveDA: a remote sensing land-cover dataset for domain adaptive semantic segmentation. arXiv preprint arXiv:2110.08733. https://doi.org/10.48550/arXiv.2110.08733

  38. Hoyer L, Dai D, Van Gool L (2022) Daformer: improving network architectures and training strategies for domain-adaptive semantic segmentation. Proceedings of Computer Vision and Pattern Recognition pp 9924–9935. https://doi.org/10.1109/CVPR52688.2022.00969

  39. Chen J, Lu Y, Yu Q, Luo X, Zhou Y (2021) Transunet: transformers make strong encoders for medical image segmentation. arXiv Preprint arXiv:2102 04306. https://doi.org/10.48550/arXiv.2102.04306

  40. Wang X, Jin Y, Long M, Wang J, Jordan MI (2019) Transferable normalization: towards improving transferability of deep neural networks. Adv Neural Inf Process Syst 32175:1953–1963. https://doi.org/10.5555/3454287.3454462

    Article  MATH  Google Scholar 

  41. Mirfallah Lialestani SP, Parcerisa D, Himi M, Abbaszadeh Shahri A (2022) Generating 3D geothermal maps in Catalonia, Spain using a hybrid adaptive multitask deep learning procedure. Energies 15(13):4602. https://doi.org/10.3390/en15134602

    Article  Google Scholar 

  42. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J big data 6(1):1–48. https://doi.org/10.1186/s40537-019-0197-0

    Article  MATH  Google Scholar 

  43. Zhang K, Chen J, Zhang T, Zhou Z (2020) A compact convolutional neural network augmented with multiscale feature extraction of acquired monitoring data for mechanical intelligent fault diagnosis. J Manuf Syst 55:273–284. https://doi.org/10.1016/j.jmsy.2020.04.016

    Article  MATH  Google Scholar 

  44. Liu Z et al (2021) Swin transformer: hierarchical vision transformer using shifted windows. Proceedings of international conference on computer vision pp 10012–10022. https://doi.org/10.1109/ICCV48922.2021.00986

  45. Zhong Z, Liu X, Jiang J, Zhao D, Ji X (2023) Deep attentional guided image filtering. Ieee Trans Neural Networks Learn Syst 1–15:1. https://doi.org/10.1109/TNNLS.2023.3253472

    Article  Google Scholar 

  46. Zou BJ et al (2018) 3D filtering by block matching and convolutional neural network for image denoising. J Comput Sci Technol 33:838–848. https://doi.org/10.1007/s11390-018-1859-7

    Article  MATH  Google Scholar 

  47. Skorokhodov I, Tulyakov S, Elhoseiny M (2022) Stylegan-v:a continuous video generator with the price, image quality and perks of stylegan2. Proc Comput Vis Pattern Recognit 3626–3636. https://doi.org/10.1109/CVPR52688.2022.00361

Download references

Funding

This research was supported by both the National Natural Science Foundation of China (Grant No. 52175471) and the Natural Science Foundation of Shandong Province (ZR2021MF110).

Author information

Authors and Affiliations

  1. School of Information and Control Engineering, Qingdao University of Technology, Qingdao, Shandong, China

    Jinlei Wang, Chengjun Chen & Chenggang Dai

  2. School of Electromechanical Engineering, Qingdao University of Science and Technology, Qingdao, Shandong, China

    Chengjun Chen

Authors
  1. Jinlei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chengjun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chenggang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Conception of project, Chengjun Chen; execution, Chengjun Chen and Jinlei Wang.; manuscript preparation, JinleiWang and Chenggang Dai.; writing of the first draft, JinleiWang and Chenggang Dai. All authors have read and agreed to the published version of the manuscript. All authors have read and agreed to the published version of the manuscript.

Corresponding author

Correspondence to Chengjun Chen.

Ethics declarations

Ethical and informed consent for data used

All data used in this paper conforms to the Ethical and informed consent specification.

Competing interests

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, J., Chen, C. & Dai, C. Domain adaptive segmentation method for mechanical assembly based on iterative loops. Appl Intell 55, 73 (2025). https://doi.org/10.1007/s10489-024-05931-y

Download citation

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10489-024-05931-y

Keywords

Search

Navigation