More Web Proxy on the site http://driver.im/

research-article

Heterogeneous domain adaptation with statistical distribution alignment and progressive pseudo label selection

Authors:

Naimeh Alipour,

Jafar TahmoresnezhadAuthors Info & Claims

Applied Intelligence, Volume 52, Issue 7

Pages 8038 - 8055

https://doi.org/10.1007/s10489-021-02756-x

Published: 01 May 2022 Publication History

Abstract

Heterogeneous domain adaptation (HDA) aims to adapt a trained model on a source domain with different input feature space to an unlabeled target domain. In fact, HDA is a challenging issue, since there exists feature and distribution discrepancies across domains. In this paper, we propose a novel approach named as heterogeneous domain adaptation with statistical distribution alignment and progressive pseudo label selection (SDA-PPLS). SDA-PPLS learns two projection matrices for source and target domains to map them into a latent subspace to have a shared feature space. Moreover, to mitigate the distribution gap, SDA-PPLS aligns both first-order and second-order statistical information, simultaneously, to improve the target classification model performance. In addition, to discriminate instances into distinct classes, SDA-PPLS aligns the class conditional distributions by pseudo label refinement of target domain data. Finally, to prevent the propagation of inaccurate pseudo labels to the next iteration, a progressive technique is proposed to select instances with higher probability. Experimental results on several real-word datasets on image to image, text to text and text to image tasks with different feature representations, demonstrate that the proposed method outperforms other state-of-the-art HDA methods.

References

[1]

Zhang J, Li W, Ogunbona P (2017) Joint geometrical and statistical alignment for visual domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 1859–1867

[2]

Saray SN and Tahmoresnezhad J Joint distinct subspace learning and unsupervised transfer classification for visual domain adaptation Signal Image Vid Process 2021 15 2 279-87

[3]

Tang B, He H, Baggenstoss PM, and Kay S A Bayesian classification approach using class-specific features for text categorization IEEE Trans Knowl Data Eng 2016 28 6 1602-1606

[4]

Dhar A, Mukherjee H, Dash NS, and Roy K Text categorization: past and present Artif Intell Rev 2021 54 4 3007-3054

[5]

Sharma R, Menghani B, Thanvi A, Kumar M, and Jangir SK An approach toward information retrieval through machine learning and its algorithms: a review Rising Threats Expert Appl Sol 2021 1187 643-649

[6]

Wang J, Feng W, Chen Y, Yu H, Huang M, Yu PS (2018) Visual domain adaptation with manifold embedded distribution alignment. In: Proceedings of the 26th ACM international conference on Multimedia. pp 402–410

[7]

Csurka G (2017) A comprehensive survey on domain adaptation for visual applications. In: Domain adaptation in computer vision applications. Springer, pp 1–35

[8]

Pereira LA and da Silva Torres R Semi-supervised transfer subspace for domain adaptation Pattern Recogn 2018 75 235-249

[9]

Ahmadvand M and Tahmoresnezhad J Metric transfer learning via geometric knowledge embedding Appl Intell 2021 51 2 921-934

[10]

Day O and Khoshgoftaar TM A survey on heterogeneous transfer learning J Big Data 2017 4 1 29

[11]

Luo Y, Wen Y, and Tao D Heterogeneous multitask metric learning across multiple domains IEEE Trans Neural Netw Learn Syst 2017 29 9 4051-4064

[12]

Ren C-X, Feng J, Dai D-Q, Yan S (2019) Heterogeneous domain adaptation via covariance structured feature translators. IEEE Transactions on Cybernetics

[13]

Li H, Pan SJ, Wan R, Kot AC (2019) Heterogeneous transfer learning via deep matrix completion with adversarial kernel embedding. In: Proceedings of the AAAI conference on artificial intelligence. pp 8602–8609

[14]

Huang J, Zhou Z, Shang J, and Niu C Heterogeneous domain adaptation with label and structural consistency Multimed Tools Appl 2020 79 25/26 17923-17943

[15]

Kadhim AI Survey on supervised machine learning techniques for automatic text classification Artif Intell Rev 2019 52 1 273-292

[16]

Becker K, Moreira VP, and dos Santos AG Multilingual emotion classification using supervised learning: Comparative experiments Inf Process Manag 2017 53 3 684-704

[17]

Hoffman J, Tzeng E, Darrell T, Saenko K (2017) Simultaneous deep transfer across domains and tasks. In: Domain adaptation in computer vision applications. Springer, pp 173–187

[18]

Zhao J, Li L, Deng F, He H, Chen J (2020) Discriminant geometrical and statistical alignment with density peaks for domain adaptation. IEEE Trans Cybern

[19]

Li S, Song S, and Huang G Prediction reweighting for domain adaptation IEEE Trans Neural Netw Learn Syst 2016 28 7 1682-1695

[20]

Huang J, Gretton A, Borgwardt K, Schölkopf B, and Smola A Correcting sample selection bias by unlabeled data Adv Neural Inform Process Syst 2006 19 601-608

[21]

Liu H, Shao M, Ding Z, and Fu Y Structure-preserved unsupervised domain adaptation IEEE Trans Knowl Data Eng 2018 31 4 799-812

[22]

Li L, Wan Z, He H (2020) Dual alignment for partial domain adaptation. IEEE Trans Cybern

[23]

Gholenji E and Tahmoresnezhad J Joint discriminative subspace and distribution adaptation for unsupervised domain adaptation Appl Intell 2020 50 7 2050-2066

[24]

Zhang W, Ouyang W, Li W, Xu D (2018) Collaborative and adversarial network for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 3801–3809

[25]

Yang B, Ma AJ, and Yuen PC Learning domain-shared group-sparse representation for unsupervised domain adaptation Pattern Recogn 2018 81 615-632

[26]

Cao Y, Long M, Wang J (2018) Unsupervised domain adaptation with distribution matching machines. In: AAAI. pp 2795–2802

[27]

Ding Z, Nasrabadi NM, and Fu Y Semi-supervised deep domain adaptation via coupled neural networks IEEE Trans Image Process 2018 27 11 5214-5224

[28]

Rezaei S, Tahmoresnezhad J, and Solouk V A transductive transfer learning approach for image classification Int J Mach Learn Cybern 2021 12 3 747-762

[29]

Gholenji E and Tahmoresnezhad J Joint local and statistical discriminant learning via feature alignment Signal Image Vid Process 2020 14 3 609-616

[30]

Li S, Liu C H, Su L, Xie B, Ding Z, Chen CP, Wu D (2020) Discriminative transfer feature and label consistency for cross-domain image classification. IEEE Trans Neural Netw Learn Syst

[31]

Tahmoresnezhad J and Hashemi S Visual domain adaptation via transfer feature learning Knowl Inf Syst 2017 50 2 585-605

[32]

Rahman MM, Fookes C, Baktashmotlagh M, and Sridharan S Correlation-aware adversarial domain adaptation and generalization Pattern Recogn 2020 100 107124

[33]

Ghifary M, Balduzzi D, Kleijn WB, and Zhang M Scatter component analysis: A unified framework for domain adaptation and domain generalization IEEE Trans Pattern Anal Mach Intell 2016 39 7 1414-1430

[34]

Lu H, Shen C, Cao Z, Xiao Y, and van den Hengel A An embarrassingly simple approach to visual domain adaptation IEEE Trans Image Process 2018 27 7 3403-3417

[35]

Li J, Lu K, Zhu L, Li Z (2017) Locality-constrained transfer coding for heterogeneous domain adaptation. In: Australasian database conference. Springer, pp 193–204

[36]

Niu C, Shang J, Huang J, Wang Y, Zhou Z (2018) Structure consistency and class discriminative feature learning for heterogeneous domain adaptation. In: 2018 IEEE 3rd international conference on communication and information systems (ICCIS). IEEE, pp 217–221

[37]

Chen C, Fu Z, Chen Z, Jin S, Cheng Z, Jin X, Hua X-S (2020) HoMM: Higher-order moment matching for unsupervised domain adaptation. In: Proceedings of the AAAI conference on artificial intelligence, vol 04, pp 3422–3429

[38]

Xu J, Ye P, Li Q, Du H, Liu Y, and Doermann D Blind image quality assessment based on high order statistics aggregation IEEE Trans Image Process 2016 25 9 4444-4457

[39]

Sun B, Feng J, Saenko K (2017) Correlation alignment for unsupervised domain adaptation. In: Domain adaptation in computer vision applications. Springer, pp 153–171

[40]

Zhang W, Zhang X, Lan L, and Luo Z Maximum mean and covariance discrepancy for unsupervised domain adaptation Neural Process Lett 2020 51 1 347-366

[41]

Xie X, Sun S, Chen H, and Qian J Domain adaptation with twin support vector machines Neural Process Lett 2018 48 2 1213-1226

[42]

Zhu L, Zhang X, Zhang W, Huang X, Guan N, Luo Z (2017) Unsupervised domain adaptation with joint supervised sparse coding and discriminative regularization term. In: 2017 IEEE international conference on image processing (ICIP). IEEE, pp 3066–3070

[43]

Hoffman J, Rodner E, Donahue J, Darrell T, Saenko K (2013) Efficient learning of domain-invariant image representations. arXiv:13013224

[44]

Li W, Duan L, Xu D, and Tsang IW Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation IEEE Trans Pattern Anal Mach Intell 2013 36 6 1134-1148

[45]

Hoffman J, Rodner E, Donahue J, Kulis B, and Saenko K Asymmetric and category invariant feature transformations for domain adaptation Int J Comput Vis 2014 109 1-2 28-41

[46]

Shi X, Liu Q, Fan W, Philip SY, Zhu R (2010) Transfer learning on heterogenous feature spaces via spectral transformation. In: 2010 IEEE international conference on data mining. IEEE, pp 1049–1054 ure

[47]

Wang C (2011) Mahadevan S Heterogeneous domain adaptation using manifold alignment. In: IJCAI proceedings-international joint conference on artificial intelligence, vol 1, p 1541

[48]

Yao Y, Zhang Y, Li X, and Ye Y Discriminative distribution alignment: A unified framework for heterogeneous domain adaptation Pattern Recogn 2020 101 107165

[49]

Fang W-C and Chiang Y-T A discriminative feature mapping approach to heterogeneous domain adaptation Pattern Recogn Lett 2018 106 13-19

[50]

Fukunaga K (2013) Introduction to statistical pattern recognition. Elsevier

[51]

Yan S, Xu D, Zhang B, Zhang H-J, Yang Q, and Lin S Graph embedding and extensions: A general framework for dimensionality reduction IEEE Trans Pattern Anal Mach Intell 2006 29 1 40-51

[52]

Tsai HY-H, Yeh Y-R, Wang FY-C (2016) Learning cross-domain landmarks for heterogeneous domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 5081–5090

[53]

Huang K, Wen H, Zhou C, Yang C, and Gui W Transfer dictionary learning method for cross-domain multimode process monitoring and fault isolation IEEE Trans Instrum Meas 2020 69 11 8713-8724

[54]

Huang K, Wu Y, Wang C, Xie Y, Yang C, and Gui W A projective and discriminative dictionary learning for high-dimensional process monitoring with industrial applications IEEE Trans Industr Inf 2020 17 1 558-568

[55]

Li J, Lu K, Huang Z, Zhu L, and Shen HT Transfer independently together: A generalized framework for domain adaptation IEEE Trans Cybern 2018 49 6 2144-2155

[56]

Shen C, Guo Y (2018) Unsupervised heterogeneous domain adaptation with sparse feature transformation. In: Asian conference on machine learning. pp 375–390

[57]

Li J, Lu K, Huang Z, Zhu L, and Shen HT Heterogeneous domain adaptation through progressive alignment IEEE Trans Neural Netw Learn Syst 2018 30 5 1381-1391

[58]

Zou Y, Yu Z, Kumar B, Wang J (2018) Unsupervised domain adaptation for semantic segmentation via class-balanced self-training. In: Proceedings of the European conference on computer vision (ECCV). pp 289–305

[59]

Wang Q, Breckon T (2020) Unsupervised domain adaptation via structured prediction based selective pseudo-labeling. In: Proceedings of the AAAI conference on artificial intelligence, vol 04, pp 6243–6250

[60]

Liang J, He R, Sun Z, and Tan T Exploring uncertainty in pseudo-label guided unsupervised domain adaptation Pattern Recogn 2019 96 106996

[61]

Wang Q, Bu P, Breckon TP (2019) Unifying unsupervised domain adaptation and zero-shot visual recognition. In: 2019 International joint conference on neural networks (IJCNN). IEEE, pp 1–8

[62]

Chen C, Xie W, Huang W, Rong Y, Ding X, Huang Y, Xu T, Huang J (2019) Progressive feature alignment for unsupervised domain adaptation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. pp 627–636

[63]

Yao Y, Zhang Y, Li X, Ye Y (2019) Heterogeneous domain adaptation via soft transfer network. In: Proceedings of the 27th ACM international conference on multimedia. pp 1578– 1586

[64]

Gretton A, Borgwardt KM, Rasch MJ, Schölkopf B, and Smola A A kernel two-sample test J Mach Learn Res 2012 13 1 723-773

[65]

Belkin M, Niyogi P, and Sindhwani V Manifold regularization: A geometric framework for learning from labeled and unlabeled examples Jo Mach Learning Res 2006 7 Nov 2399-2434

[66]

Lee H, Battle A, Raina R, and Ng A Efficient sparse coding algorithms Adv Neural Inf Process Syst 2006 19 801-808

[67]

Li J, Zhao J, Lu K (2016) Joint feature selection and structure preservation for domain adaptation. In: IjCAI. pp 1697–1703

[68]

Berlind C, Urner R (2015) Active nearest neighbors in changing environments. In: International conference on machine learning. pp 1870–1879

[69]

Gentleman R, Carey VJ (2008) Unsupervised machine learning. In: Bioconductor case studies. Springer, pp 137–157

[70]

Saenko K, Kulis B, Fritz M, Darrell T (2010) Adapting visual category models to new domains. In: European conference on computer vision. Springer, pp 213–226

[71]

Griffin G, Holub A, Perona P (2007) Caltech-256 object category dataset

[72]

Bay H, Ess A, Tuytelaars T, and Van Gool L Speeded-up robust features (SURF) Comput Vis Image Underst 2008 110 3 346-359

[73]

Donahue J, Jia Y, Vinyals O, Hoffman J, Zhang N, Tzeng E, Darrell T (2014) Decaf: A deep convolutional activation feature for generic visual recognition. In: International conference on machine learning. pp 647–655

[74]

Amini MR, Usunier N, Goutte C (2009) Learning from multiple partially observed views-an application to multilingual text categorization. In: Advances in neural information processing systems. pp 28–36

[75]

Tommasi T, Tuytelaars T (2014) A testbed for cross-dataset analysis. In: European conference on computer vision. Springer, pp 18–31

[76]

Chua T-S, Tang J, Hong R, Li H, Luo Z, Zheng Y (2009) NUS-WIDE: a real-world web image database from national university of singapore. In: Proceedings of the ACM international conference on image and video retrieval. pp 1–9

[77]

Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248–255

[78]

Chen W-Y, Hsu T-MH, Tsai Y-HH, Wang Y-CF, Chen M-S (2016) Transfer neural trees for heterogeneous domain adaptation. In: European conference on computer vision. Springer, pp 399–414

[79]

Li W, Duan L, Xu D, and Tsang IW Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation IEEE Trans Pattern Anal Mach Intell 2013 36 6 1134-1148

[80]

Li J, Jing M, Lu K, Zhu L, and Shen HT Locality preserving joint transfer for domain adaptation IEEE Trans Image Process 2019 28 12 6103-6115

Cited By

Wang YXu YYang JWu MLi XXie LChen Z(2024)SEA++: Multi-Graph-Based Higher-Order Sensor Alignment for Multivariate Time-Series Unsupervised Domain AdaptationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.344490446:12(10781-10796)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3444904
Gentner NSusto G(2024)Heterogeneous domain adaptation and equipment matchingComputers and Industrial Engineering10.1016/j.cie.2023.109821187:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.cie.2023.109821
Liu YLi LTan JRao YTan XLi Y(2024)Cross-domain Fisher Discrimination Criterion: A Domain Adaptive Method Based on the Nature of ClassifierApplied Intelligence10.1007/s10489-024-05376-354:7(5389-5405)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05376-3
Show More Cited By

Index Terms

Heterogeneous domain adaptation with statistical distribution alignment and progressive pseudo label selection
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
    1. Learning paradigms
    2. Machine learning approaches

Index terms have been assigned to the content through auto-classification.

Recommendations

Heterogeneous domain adaptation by semantic distribution alignment network
Abstract
Domain adaptation leverages the knowledge in the source domain to build a model that is capable of performing effectively in the target domain. Heterogeneous domain adaptation (HDA) is a special case of domain adaptation to deal with the situation ...
Cross-domain structure preserving projection for heterogeneous domain adaptation
Highlights
- Extending locality preserving projection (LPP) to multi-domain scenarios.
- ...
Abstract
Heterogeneous Domain Adaptation (HDA) addresses the transfer learning problems where data from the source and target domains are of different modalities (e.g., texts and images) or feature dimensions (e.g., features extracted with ...
Label-guided heterogeneous domain adaptation
Abstract
Heterogeneous domain adaptation(HDA) mainly considers how to solve the target domain task with the help of the relevant knowledge of the source domain when both data distribution and feature space of the target domain are different from the source ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Applied Intelligence

Applied Intelligence Volume 52, Issue 7

May 2022

1300 pages

ISSN:0924-669X

Issue’s Table of Contents

© The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature 2021.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 May 2022

Accepted: 09 August 2021

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YXu YYang JWu MLi XXie LChen Z(2024)SEA++: Multi-Graph-Based Higher-Order Sensor Alignment for Multivariate Time-Series Unsupervised Domain AdaptationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.344490446:12(10781-10796)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3444904
Gentner NSusto G(2024)Heterogeneous domain adaptation and equipment matchingComputers and Industrial Engineering10.1016/j.cie.2023.109821187:COnline publication date: 12-Apr-2024
https://dl.acm.org/doi/10.1016/j.cie.2023.109821
Liu YLi LTan JRao YTan XLi Y(2024)Cross-domain Fisher Discrimination Criterion: A Domain Adaptive Method Based on the Nature of ClassifierApplied Intelligence10.1007/s10489-024-05376-354:7(5389-5405)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s10489-024-05376-3
Chen YZhou HWang ZZhong P(2024)Heterogeneous domain adaptation by class centroid matching and local discriminative structure preservationNeural Computing and Applications10.1007/s00521-024-09786-936:21(12865-12881)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1007/s00521-024-09786-9
V JS A(2023)Cross-lingual Sentiment Analysis of Tamil Language Using a Multi-stage Deep Learning ArchitectureACM Transactions on Asian and Low-Resource Language Information Processing10.1145/363139122:12(1-28)Online publication date: 1-Nov-2023
https://dl.acm.org/doi/10.1145/3631391
Wolleb JSandkühler RBieder FBarakovic MHadjikhani NPapadopoulou AYaldizli ÖKuhle JGranziera CCattin P(2022)Learn to Ignore: Domain Adaptation for Multi-site MRI AnalysisMedical Image Computing and Computer Assisted Intervention – MICCAI 202210.1007/978-3-031-16449-1_69(725-735)Online publication date: 18-Sep-2022
https://dl.acm.org/doi/10.1007/978-3-031-16449-1_69

View Options

View options

Media

Figures

Other

Tables

View Issue’s Table of Contents