[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ Skip to main content
Springer Nature Link
Log in

Improved multi object tracking with locality sensitive hashing

  • Industrial and Commercial Application
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Object tracking is one of the most advanced applications of computer vision algorithms. While various tracking approaches have been previously developed, they often use many approximations and assumptions to enable real-time performance within the resource constraints in terms of memory, time and computational requirements. In order to address these limitations, we investigate the bottlenecks of existing tracking frameworks and propose a solution to enhance tracking efficiency. The proposed method uses Locality Sensitive Hashing (LSH) to efficiently store and retrieve nearest neighbours and then utilizes a bipartite cost matching based on the predicted positions, size, aspect ratio, appearance description, and uncertainty in motion estimation. The LSH algorithm helps reduce the dimensionality of the data while preserving their relative distances. LSH hashes the features in constant time and facilitates rapid nearest neighbour retrieval by considering features falling into the same hash buckets as similar. The effectiveness of the method was evaluated on the MOT benchmark dataset and achieved Multiple Object Tracker Accuracy (MOTA) of 67.1% (train) and 62.7% (test). Furthermore, our framework exhibits the highest Multiple Object Tracker Precision (MOTP), mostly tracked objects, and the lowest values for mostly lost objects and identity switches among the state-of-the-art trackers. The incorporation of LSH implementation reduced identity switches by approximately 7% and fragmentation by around 13%. We used the framework for real-time tracking applications on edge devices for an industry partner. We found that the LSH integration resulted in a notable reduction in track ID switching, with only a marginal increase in computation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
£29.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price includes VAT (United Kingdom)

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1

Similar content being viewed by others

Data availability

Supplementary materials that deeply analyse the results are given in the appendix section.

Notes

  1. https://github.com/ajaichemmanam/OTLSH-Tracker.

  2. https://github.com/cheind/py-motmetrics.

References

  1. Bewley A, Ge Z, Ott L, Ramos F, Upcroft B (2016) Simple online and realtime tracking. In: 2016 IEEE international conference on image processing (ICIP). IEEE, pp 3464–3468

  2. Yang M, Han G, Yan B, Zhang W, Qi J, Lu H, Wang D (2024) Hybrid-sort: weak cues matter for online multi-object tracking. In: Proceedings of the AAAI conference on artificial intelligence, vol 38, pp 6504–6512

  3. Cao J, Pang J, Weng X, Khirodkar R, Kitani K (2023) Observation-centric sort: Rethinking sort for robust multi-object tracking. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9686–9696

  4. Zhang Y, Sun P, Jiang Y, Yu D, Weng F, Yuan Z, Luo P, Liu W, Wang X (2022) Bytetrack: multi-object tracking by associating every detection box. In: European conference on computer vision. Springer, pp 1–21

  5. Zhang Y, Wang C, Wang X, Zeng W, Liu W (2020) A simple baseline for multi-object tracking. arXiv preprint arXiv:2004.01888

  6. Wojke N, Bewley A, Paulus D (2017) Simple online and realtime tracking with a deep association metric. In: 2017 IEEE international conference on image processing (ICIP). IEEE, pp 3645–3649

  7. Karunasekera H, Wang H, Zhang H (2019) Multiple object tracking with attention to appearance, structure, motion and size. IEEE Access 7:104423–104434

    Article  Google Scholar 

  8. Zhou X, Koltun V, Krähenbühl P (2020) Tracking objects as points. In: European conference on computer vision. Springer, pp 474–490

  9. Yang K, He Z, Pei W, Zhou Z, Li X, Yuan D, Zhang H (2021) Siamcorners: siamese corner networks for visual tracking. IEEE Trans Multimedia 24:1956–1967

    Article  Google Scholar 

  10. Tang S, Andriluka M, Andres B, Schiele B (2017) Multiple people tracking by lifted multicut and person re-identification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3539–3548

  11. Sadeghian A, Alahi A, Savarese S (2017) Tracking the untrackable: Learning to track multiple cues with long-term dependencies. In: Proceedings of the IEEE international conference on computer vision, pp 300–311

  12. Ning G, Zhang Z, Huang C, He Z, Ren X, Wang H (2016) Spatially supervised recurrent convolutional neural networks for visual object tracking. arXiv preprint arXiv:1607.05781

  13. Yuan D, Chang X, Huang P-Y, Liu Q, He Z (2020) Self-supervised deep correlation tracking. IEEE Trans Image Process 30:976–985

    Article  Google Scholar 

  14. Wang Z, Zheng L, Liu Y, Wang S (2019) Towards real-time multi-object tracking. arXiv preprint arXiv:1909.12605

  15. Tsai C-Y, Shen G-Y, Nisar H (2023) Swin-jde: joint detection and embedding multi-object tracking in crowded scenes based on swin-transformer. Eng Appl Artif Intell 119:105770

    Article  Google Scholar 

  16. Chemmanam AJ, Jose BA, Moopan A (2023) A multi-tasking model for object detection, instance segmentation and keypoint estimation tasks. J Inf Sci Eng 39(3)

  17. Chemmanam AJ, Jose BA (2022) Joint learning for multitasking models. In: Responsible data science: select proceedings of ICDSE 2021. Springer, Singapore, pp 155–167

  18. Zhao Z, Wang J, Horn M, Ding Y, He T, Bai Z, Zietlow D, Simon-Gabriel C-J, Shuai B, Tu Z, et al (2023) Object-centric multiple object tracking. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 16601–16611

  19. Ding Z, Liu S, Li M, Lian Z, Xu H (2020) A blockchain-enabled multiple object tracking for unmanned system with deep hash appearance feature. IEEE Access 9:1116–1123

    Article  Google Scholar 

  20. Wei H, Huang Y (2022) Online multiple object tracking using spatial pyramid pooling hashing and image retrieval for autonomous driving. Machines 10(8):668

    Article  Google Scholar 

  21. Zha C, Luo S, Xu X (2024) Infrared multi-target detection and tracking in dense urban traffic scenes. IET Image Process

  22. Shi N, Fu C, Tie M, Zhang W, Wang X, Sham C-W (2024) Attention-based deep supervised hashing for near duplicate video retrieval. Neural Comput Appl 36(10):5217–5230

    Article  Google Scholar 

  23. Ghasemi M, Hassanpour H (2024) Frih: a face recognition framework using image hashing. Multimedia Tools Appl 1–23

  24. Bodla N, Singh B, Chellappa R, Davis LS (2017) Soft-nms–improving object detection with one line of code. In: Proceedings of the IEEE international conference on computer vision, pp 5561–5569

  25. Bochinski E, Eiselein V, Sikora T (2017) High-speed tracking-by-detection without using image information. In: 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6

  26. Zheng L, Shen L, Tian L, Wang S, Wang J, Tian Q (2015) Scalable person re-identification: a benchmark. In: Proceedings of the IEEE international conference on computer vision

  27. Zheng L, Bie Z, Sun Y, Wang J, Su C, Wang S, Tian Q (2016) MARS: a video benchmark for large-scale person re-identification. Springer

  28. Kuhn HW (1955) The Hungarian method for the assignment problem. Naval Res Logist Q 2(1–2):83–97

    Article  MathSciNet  Google Scholar 

  29. Jonker R, Volgenant A (1987) A shortest augmenting path algorithm for dense and sparse linear assignment problems. Computing 38(4):325–340

    Article  MathSciNet  Google Scholar 

  30. Indyk P, Motwani R (1998) Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the 30th annual ACM symposium on theory of computing, pp 604–613

  31. Jing Y, Baluja S (2008) Visualrank: applying pagerank to large-scale image search. IEEE Trans Pattern Anal Mach Intell 30(11):1877–1890

    Article  Google Scholar 

  32. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vis 60(2):91–110

    Article  Google Scholar 

  33. Milan A, Leal-Taixé L, Reid I, Roth S, Schindler K (2016) Mot16: a benchmark for multi-object tracking. arXiv preprint arXiv:1603.00831

  34. Nithin PB, Francis A, Chemmanam AJ, Jose BA, Mathew J (2019) Face tracking robot testbed for performance assessment of machine learning techniques. In: 2019 7th International conference on smart computing communications (ICSCC), pp 1–5

  35. Nithin PB, Francis A, Chemmanam AJ, Jose BA, Mathew J (2020) Interactive robotic testbed for performance assessment of machine learning based computer vision techniques. Special Issue on Smart Computational Intelligence for a Digital World - Journal of Information Science and Engineering (JISE) 36(5)

  36. Yu F, Li W, Li Q, Liu Y, Shi X, Yan J (2016) Poi: multiple object tracking with high performance detection and appearance feature. In: European conference on computer vision. Springer, pp 36–42

  37. Zhou X, Wang D, Krähenbühl P (2019) Objects as points. arXiv Preprint arXiv:1904.07850

  38. Keuper M, Tang S, Zhongjie Y, Andres B, Brox T, Schiele B (2016) A multi-cut formulation for joint segmentation and tracking of multiple objects. arXiv preprint arXiv:1607.06317

  39. Lee B, Erdenee E, Jin S, Nam MY, Jung YG, Rhee PK (2016) Multi-class multi-object tracking using changing point detection. In: European conference on computer vision. Springer, pp 68–83

  40. Choi W (2015) Near-online multi-target tracking with aggregated local flow descriptor. In: Proceedings of the IEEE international conference on computer vision, pp 3029–3037

  41. Sanchez-Matilla R, Poiesi F, Cavallaro A (2016) Online multi-target tracking with strong and weak detections. In: European conference on computer vision. Springer, pp 84–99

Download references

Acknowledgements

This research work is supported by Vuelogix Technologies Pvt Ltd, Confederation of Indian Industry (CII) and Department of Science and Technology, Govt of India (DST-SERB), through Prime Minister’s Fellowship for Doctoral Research 2020.

Author information

Author notes

  1. Bijoy Jose and Asif Moopan have contributed equally to this work.

Authors and Affiliations

  1. Cyber Physical Systems Lab, Cochin University of Science and Technology, Kochi, Kerala, 682022, India

    Ajai John Chemmanam & Bijoy Jose

  2. Vuelogix Technologies Pvt. Ltd., Kochi, Kerala, 682037, India

    Asif Moopan

Authors
  1. Ajai John Chemmanam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bijoy Jose
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Asif Moopan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bijoy Jose.

Ethics declarations

Conflict of interest

The authors have no other conflict of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file 1 (pdf 93 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chemmanam, A.J., Jose, B. & Moopan, A. Improved multi object tracking with locality sensitive hashing. Pattern Anal Applic 27, 136 (2024). https://doi.org/10.1007/s10044-024-01353-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10044-024-01353-1

Keywords

Search

Navigation