[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to main content

Deep Learning Approach for Enhanced Object Recognition and Assembly Guidance with Augmented Reality

  • Conference paper
  • First Online:
Intelligent Human Computer Interaction (IHCI 2023)

Abstract

In an effort to enhance the efficiency and precision of manual part assembly in industrial settings, the development of software for assembly guidance becomes imperative. Augmented reality (AR) technology offers a means to provide visual instructions for assembly tasks, rendering the guidance more comprehensible. Nevertheless, a significant challenge lies in the technology’s limited object detection capabilities, especially when distinguishing between similar assembled parts. This project proposes the utilization of deep learning neural networks to enhance the accuracy of object recognition within the AR guided assembly application. To achieve this objective, a dataset of assembly parts, known as the Visual Object Classes (VOC) dataset, was created. Data augmentation techniques were employed to expand this dataset, incorporating scale HSV (hue saturation value) transformations. Subsequently, deep learning models for the recognition of assembly parts were developed which were based on the Single Shot Multibox Detector (SSD) and the YOLOv7 detector. The models were trained and fine-tuned, targeting on the variations of the positions of detected parts. The effectiveness of this approach was evaluated using a case study involving an educational electronic blocks circuit science kit. The results demonstrated a high assembly part recognition accuracy of over 99% in mean average precision (MAP), along with favorable user testing outcomes. Consequently, the AR application was capable of offering high-quality guidance to users which holds promise for application in diverse scenarios and the resolution of real-world challenges.

This work was supported in part by the Zhejiang Provincial Natural Science Foundation of China under Grant LQ21F020024, and in part by the Ningbo Science and Technology (S&T) Bureau through the Major S &T Program under Grant 2021Z037 and 2022Z080.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
GBP 19.95
Price includes VAT (United Kingdom)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
GBP 49.99
Price includes VAT (United Kingdom)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
GBP 59.99
Price includes VAT (United Kingdom)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., MacIntyre, B.: Recent advances in augmented reality. IEEE Comput. Graphics Appl. 21(6), 34–47 (2001)

    Article  Google Scholar 

  2. Lindner, C., Rienow, A., Jürgens, C.: Augmented reality applications as digital experiments for education - an example in the earth-moon system. Acta Astronaut. 161, 66–74 (2019)

    Article  Google Scholar 

  3. Doshi, A., Smith, R.T., Thomas, B.H., Bouras, C.: Use of projector based augmented reality to improve manual spot-welding precision and accuracy for automotive manufacturing. Int. J. Adv. Manuf. Technol. 89(5–8), 1279–1293 (2017)

    Article  Google Scholar 

  4. Jiang, S., Ong, S.K., Nee, A.Y.C.: An AR-based hybrid approach for facility layout planning and evaluation for existing shop floors. Int. J. Adv. Manuf. Technol. 72(1–4), 457–473 (2014)

    Article  Google Scholar 

  5. Katiyar, A., Kalra, K., Garg, C.: Marker based augmented reality. Adv. Comput. Sci. Inf. Technol. (ACSIT) 2(5), 441–445 (2015). Cited by: 43

    Google Scholar 

  6. Vipul Sharma and Roohie Naaz Mir: A comprehensive and systematic look up into deep learning based object detection techniques: a review. Comput. Sci. Rev. 38, 100301 (2020)

    Article  MathSciNet  Google Scholar 

  7. Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: unified, real-time object detection (2015)

    Google Scholar 

  8. Wang, C.-Y., Bochkovskiy, A., Liao, H.-Y.M.: YOLOv7: trainable bag-of-freebies sets new state-of-the-art for real-time object detectors (2022)

    Google Scholar 

  9. Liu, W., et al.: SSD: single shot multibox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  10. Brooke, J.: SUS: a quick and dirty usability scale. Usab. Eval. Ind. 189, 11 (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Boon Giin Lee .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lee, B.G., Wang, X., Han, R., Sun, L., Pike, M., Chung, WY. (2024). Deep Learning Approach for Enhanced Object Recognition and Assembly Guidance with Augmented Reality. In: Choi, B.J., Singh, D., Tiwary, U.S., Chung, WY. (eds) Intelligent Human Computer Interaction. IHCI 2023. Lecture Notes in Computer Science, vol 14532. Springer, Cham. https://doi.org/10.1007/978-3-031-53830-8_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-53830-8_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-53829-2

  • Online ISBN: 978-3-031-53830-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics