Dual-Modality Feature Extraction Network Based on Graph Attention for RGBT Tracking
Authors: 
Zhenjie Tian, Siqing Chen, Muyang Li, Kuo Liao, Ping Zhang, Wenyao ZhaoAuthors Info & Claims
Pages 248 - 253
Abstract
The RGBT target tracking method has recently gained popularity owing to the complementarity of RGB images and thermal images information. Although numerous RGBT tracking methods have been proposed, effectively utilizing dual-modality information is still challenging. To solve this problem, we design a dual-modality feature extraction network to extract common and specific modality features. For specific modality features, we design two unique feature extraction networks to learn the independent dual-modality information respectively. For common modality features, we propose a common feature extraction network based on the graph attention method, which could learn the shared modality information of dual-modality images. According to experiments on the RGBT234 and LasHeR datasets, our suggested method performs sufficiently.
References
[1]
Hyeonseob Nam, Bohyung Han.2016. Learning Multi-domain Convolutional Neural Networks for Visual Tracking. 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA. 4293-4302
[2]
Chenglong Li, Andong Lu, Aihua Zheng, Zhengzheng Tu, Jin Tang.2019. Multi-Adapter RGBT Tracking. 2019 IEEE/CVF International Conference on Computer Vision Workshops, ICCV Workshops 2019, Seoul, Korea. 2262-2270.
[3]
Andong Lu, Chenglong Li, Yuqing Yan, Jin Tang, Bin Luo.2021. RGBT Tracking via Multi-Adapter Network with Hierarchical Divergence Loss. IEEE Trans. Image Process. 30(2021),5613-5625.
[4]
Pengyu Zhang, Jie Zhao, Chunjuan Bo, Dong Wang, Huchuan Lu, Xiaoyun Yang. 2021. Jointly Modeling Motion and Appearance Cues for Robust RGB-T Tracking. IEEE Trans. Image Process. 30(2021),3335-3347.
[5]
Xingchen Zhang, Ping Ye, Shengyun Peng, Jun Liu, Ke Gong, Gang Xiao. 2019. SiamFT: An RGB-Infrared Fusion Tracking Method via Fully Convolutional Siamese Networks. IEEE Access 7: 122122-122133 (2019).
[6]
Y . Zhu, C. Li, B. Luo, J. Tang, and X. Wang, “Dense feature aggregation and pruning for RGBT tracking,” in Proc. 27th ACM Int. Conf. Multimedia, Oct. 2019, pp. 465–472.
[7]
Luca Bertinetto, Jack Valmadre, João F. Henriques, Andrea Vedaldi, Philip H.S. Torr.2016. Fully-Convolutional Siamese Networks for Object Tracking. 14th European Conference on Computer Vision. Amsterdam, The Netherlands. 850-865.
[8]
Xingchen Zhang, Ping Ye, Shengyun Peng, Jun Liu, Gang Xiao.2020. DSiamMFT: An RGB-T fusion tracking method via dynamic Siamese networks using multi-layer feature fusion. Signal Processing: Image Communication. 84(2020): 115756
[9]
Petar V eliˇckovi´c, Guillem Cucurull, Arantxa Casanova. Graph attention networks. In Int. Conf. Learn. Represent., 2018.
[10]
Chenglong Li, Xinyan Liang, Yijuan Lu, Nan Zhao, Jin Tang. 2019. RGB-T object tracking: Benchmark and baseline. Pattern Recognition. 96(2019)
[11]
Chenglong Li, Wanlin Xue, Yaqing Jia, Zhichen Qu, Bin Luo, Jin Tang. 2021. LasHeR: A Large-scale High-diversity Benchmark for RGBT Tracking. CoRR abs/2104.13202 (2021)
[12]
Lianghua Huang, Xin Zhao, Kaiqi Huang. 2021.GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild. IEEE Transactions on Pattern Analysis and Machine Intelligence. 43,5((2021)): 1562-1577.
[13]
M. Danelljan, G. Bhat, F. S. Khan, and M. Felsberg. 2017. ECO: Efficient convolution operators for tracking. In Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2017. 6.
[14]
João F. Henriques, Rui Caseiro, Pedro Martins, Jorge Batista.2015. High-Speed Tracking with Kernelized Correlation Filters. IEEE Transactions on Pattern Analysis and Machine Intelligence .37,3(2015). 583-596.
[15]
LukeLukei A, Vojí, Tomá, Ehovinzajc L, Discriminative Correlation Filter with Channel and Spatial Reliability[J]. International Journal of Computer Vision, 2018.
[16]
Chenglong Li, Chengli Zhu, Yan Huang, Jin Tang, Liang Wang. 2018. Cross-Modal Ranking with Soft Consistency and Noisy Labels for Robust RGB-T Tracking. 15th European Conference on Computer Vision, Munich, Germany. 831-847.
[17]
Yabin Zhu, Chenglong Li, Bin Luo, Jin Tang, Xiao Wang. 2019. Dense Feature Aggregation and Pruning for RGBT Tracking. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France. 465-472.
[18]
Yabin Zhu, Chenglong Li, Jin Tang, Bin Luo. 2021. Quality-Aware Feature Aggregation Network for Robust RGBT Tracking. IEEE Transactions on Intelligent Vehicles, 6,1 (2021).121-130.
[19]
Yuan Gao, Chenglong Li, Yabin Zhu, Jin Tang, Tao He, Futian Wang. 2019. Deep Adaptive Fusion Network for High-Performance RGBT Tracking. 2019 IEEE/CVF International Conference on Computer Vision Workshops, Seoul, Korea. 91-99.
[20]
Chenglong Li, Lei Liu, Andong Lu, Qing Ji, Jin Tang. 2020. Challenge-Aware RGBT Tracking. 16th European Conference on Computer Vision, Glasgow, UK. 222-237.
Recommendations
Multi-Modal Fusion Object Tracking Based on Fully Convolutional Siamese Network
CACML '23: Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine LearningRGBT tracking incorporates thermal infrared data to achieve more accurate visual tracking. However, the efficiency of RGBT tracking may be diminished by some bottlenecks, such as thermal crossover, illumination variation and occlusion. To address the ...
QueryTrack: Joint-Modality Query Fusion Network for RGBT Tracking
Existing RGB-Thermal trackers usually treat intra-modal feature extraction and inter-modal feature fusion as two separate processes, therefore the mutual promotion of extraction and fusion is neglected. Then, the complementary advantages of RGB-T fusion ...
Feature Fusion: Graph Attention Network and CNN Combing for Hyperspectral Image Classification
ICCCV '22: Proceedings of the 5th International Conference on Control and Computer VisionGraph convolutional networks (GCNs) have attracted increasing attention in hyperspectral image classification. However, most of the available GCN-based HSI classification methods treat superpixels as graph nodes, ignoring pixel-level spectral spatial ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2022
253 pages
ISBN:9781450396851
DOI:10.1145/3562007
Copyright © 2022 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 October 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CCRIS'22
CCRIS'22: 2022 3rd International Conference on Control, Robotics and Intelligent System
August 26 - 28, 2022
Virtual Event, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 51Total Downloads
- Downloads (Last 12 months)16
- Downloads (Last 6 weeks)4
Reflects downloads up to 24 Dec 2024
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format