Transferring fashion to surveillance with weak labels
Pages 13021 - 13035
Abstract
In this paper, we address the problem of automatic clothing parsing in surveillance images using the information from user-generated tags, such as “jeans” and “T-shirt.” Although clothing parsing has achieved great success in the fashion domain, it is quite challenging to parse target under practical surveillance conditions due to the presence of complex environmental interference, such as that from low resolution, viewpoint variations and lighting changes. Our method is developed to capture target information from the fashion domain and apply this information to a surveillance domain by weakly supervised transfer learning. Most target tags convey strong location information (e.g., “T-shirt” is always shown in the upper region), which can be used as weak labels for our transfer method. Both quantitative and qualitative experiments conducted on practical surveillance datasets demonstrate the effectiveness of the proposed surveillance data enhancing method.
References
[1]
Li A, Liu L, Wang K, Liu S, and Yan S Clothing attributes assisted person reidentification IEEE Trans Circuits Syst Video Technol 2015 25 5 869-878
[2]
Wang Z, Hu R, Liang C, Yu Y, Jiang J, Ye M, Chen J, and Leng Q Zero-shot person re-identification via cross-view consistency IEEE Trans Multimed 2016 18 2 260-272
[3]
Ye M, Liang C, Yu Y, Wang Z, Leng Q, Xiao C, Chen J, and Hu R Person reidentification via ranking aggregation of similarity pulling and dissimilarity pushing IEEE Trans Multimed 2016 18 12 2553-2566
[4]
Yang J, Franco J-S, Hétroy-Wheeler F, Wuhrer S (2016) Estimation of human body shape in motion with wide clothing. In: Proceedings of the European conference on computer vision (ECCV), Amsterdam, The Netherland, pp 439–454
[5]
Weber M, Bauml M, Stiefelhagen R (2011) Part-based clothing segmentation for person retrieval. In: Proceedings of the IEEE international conference on advanced video and signal based surveillance (AVSS), Klagenfurt, Austria, pp 361–366
[6]
Simo-Serra E, Fidler S, Moreno-Noguer F, Urtasun R (2014) A high performance CRF model for clothes parsing. In: Proceedings of the Asian conference on computer vision (ACCV), Singapore, pp 64–81
[7]
Yamaguchi K, Kiapour MH, Berg TL (2013) Paper doll parsing: retrieving similar styles to parse clothing items. In: Proceedings of the IEEE international conference on computer vision (CVPR), Portland, OR, USA, pp 3519–3526
[8]
Yang W, Luo P, Lin L (2014) Clothing co-parsing by joint image segmentation and labeling. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Columbus, OH, USA, pp 3182–3189
[9]
Tangseng P, Wu Z, Yamaguchi K (2017) Looking at outfit to parse clothing. https://arxiv.org/abs/1703.01386
[10]
Bourdev L, Maji S, Malik J (2011) Describing people: a poselet-based approach to attribute classification. In: Proceedings of the IEEE conference on international conference on computer vision (ICCV), Barcelona, Spain, pp 1543–1550
[11]
Guan P, Freifeld O, Black MJ (2010) A 2D human body model dressed in Eigen clothing. In: Proceedings of the European conference on computer vision (ECCV), Heraklion, Crete, Greece, pp 285–298
[12]
Gallagher AC, Chen T (2008) Clothing cosegmentation for recognizing people. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Anchorage, AK, USA, pp 1–8
[13]
Layne R, Hospedales TM, Gong S, Mary Q (2012) Person re-identification by attributes. In: Proceedings of the British machine vision conference (BMVC), Guildfor, UK, p. 8
[14]
Koestinger M, Hirzer M, Wohlhart P, Roth PM, Bischof H (2012) Large scale metric learning from equivalence constraints. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Providence, Rhode Island, pp 2288–2295
[15]
Yang M, Yu K (2011) Real-time clothing recognition in surveillance videos. In: Proceedings of the IEEE international conference on image process. (ICIP), Brussels, Belgium, pp 2937–2940
[16]
Shi Z, Hospedales TM, Xiang T (2015) Transferring a semantic representation for person re-identification and search. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Boston, MA, USA, pp 4184–4193
[17]
Oren M, Papageorgiou C, Sinha P, Osuna E, Poggio T (1997) Pedestrian detection using wavelet templates. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), San Juan, Puerto Rico, pp 193–199
[18]
Yamaguchi K, Kiapour MH, Ortiz LE, Berg TL (2012) Parsing clothing in fashion photographs. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Providence, Rhode Island, pp 3570–3577
[19]
West J, Ventury D, Warnick S (2007) Spring research presentation: a theoretical foundation for inductive transfer. Brigham Young University, College of Physical and Mathematical Sciences
[20]
Chen Q, Huang J, Feris R, Brown LM, Dong J, Yan S (2015) Deep domain adaptation for describing people based on fine-grained clothing attributes. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Boston, MA, USA, pp 5315–5324
[21]
Xiao T, Xia T, Yang Y, Huang C, Wang X (2015) Learning from massive noisy labeled data for image classification. In: Proceedings of the IEEE conference on computer vision pattern recognition (CVPR), Boston, MA, USA, pp 2691–2699
[22]
Pan SJ and Yang Q A survey on transfer learning IEEE Trans Knowl Data Eng 2010 22 10 1345-1359
[23]
Zheng Q, Chen J, Liang C, Fang W, Jing X, Hu R (2017) Transferring clothing parsing from fashion dataset to surveillance. In: Proceedings of the IEEE international conference on acoustics speech and signal processing (ICASSP), New Orleans, LA, USA, pp 1667–1671
[24]
Lin J, Yang H, Chen D et al (2019) Face parsing with RoI Tanh-Warping. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 5654–5663
[25]
Chu W, Hung WC, Tsai YH et al (2019) Weakly-supervised caricature face parsing through domain adaptation. In: Proceedings of the IEEE international conference on image processing (ICIP). IEEE, pp 3282–3286
[26]
Zhang P, Liu W, Lei Y, et al. RAPNet: residual atrous pyramid network for importance-aware street scene parsing IEEE Trans Image Process 2020 29 5010-5021
[27]
Chen H, Xu ZJ, Liu ZQ, Zhu SC (2006) Composite templates for cloth modeling and sketching. In: Proceedings of the IEEE conference on computer vision on pattern recognition (CVPR), vol 1, New York, NY, USA, pp 943–950
[28]
Wang F, Zhao Q, Yin B, and Xu T Parsing fashion image into mid-level semantic parts based on chain-conditional random fields IET Image Process 2016 10 6 456-463
[29]
Chen D, Tang Y, Zhang H, Wang L, Li X (2019) Incremental factorization of big time series data with blind factor approximation. IEEE Trans Knowl Data Eng
[30]
Ke H, Chen D, Shi B, Zhang J, Liu X, Zhang X, and Li X Improving brain E-health services via high-performance EEG classification with grouping Bayesian optimization IEEE Trans Serv Comput 2019
[31]
Chen D, Hu Y, Wang L, Zomaya AY, and Li X H-parafac: hierarchical parallel factor analysis of multidimensional big data IEEE Trans Parallel Distrib Syst 2016 28 4 1091-1104
[32]
Ruan W, Liu W, Bao Q, Chen J, Cheng Y, Mei T (2019) Poinet: Pose-guided ovonic insight network for multi-person pose tracking. In: Proceedings of the 27th ACM international conference on multimedia. ACM, pp 284–292
[33]
Liu F, Xue S, Wu J et al (2020) Deep learning for community detection: progress, challenges and opportunities
[34]
Yang L, Rodriguez H, Crucianu M, Ferecatu M (2017) Fully convolutional network with superpixel parsing for fashion web image segmentation. In: International conference on multimedia modeling (MMM), Reykjavík, Iceland, pp 139–151
[35]
He Y, Yang L, Chen L (2017) Real-time fashion-guided clothing semantic parsing: a lightweight multi-scale inception neural network and benchmark. In: AAAI conference on artificial intelligent (AAAI), San Francisco, CA, USA
[36]
Zhou L, Wang Z, Luo Y, and Xiong Z Separability and compactness network for image recognition and superresolution IEEE Trans Neural Netw Learn Syst 2020 30 11 3275-3286
[37]
Yi P, Wang Z, Jiang K, Shao Z, and Ma J Multi-temporal ultra dense memory network for video super-resolution IEEE Trans Circuits Syst Video Technol 2020 30 8 2503-2516
[38]
Jiang K, Wang Z, Yi P, Wang G, Lu T, and Jiang J Edge-enhanced GAN for remote sensing image superresolution IEEE Trans Geosci Remote Sens 2019 57 8 5799-5812
[39]
Du B, Tang X, Wang Z, et al. Robust graph-based semisupervised learning for noisy labeled data via maximum correntropy criterion IEEE Trans Cybern 2019 49 4 1440-1453
[40]
Fernando B, Habrard A, Sebban M, Tuytelaars T (2013) Unsupervised visual domain adaptation using subspace alignment. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney, NSW, Australia, pp 2960–2967
[41]
Saenko K, Kulis B, Fritz M, Darrell T (2010), Adapting visual category models to new domains. In: Proceedings of the European conference on computer vision (ECCV), Heraklion, Crete, Greece, pp 213–226
[42]
Bian W, Tao D, and Rui Y Cross-domain human action recognition IEEE Trans Syst Man Cybern B 2012 42 2 298
[43]
Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) Cnn features off-the-shelf: an astounding baseline for recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, USA, pp 512–519
[44]
Hoffman J, Tzeng E, Donahue J, Jia Y, Saenko K, Darrell T (2014) One-shot adaptation of supervised deep convolutional models. In: Proceedings of the international conference on learning representations (ICLR), Banff, Canada
[45]
Li X, Zhang L, Du B, et al. Iterative reweighting heterogeneous transfer learning framework for supervised remote sensing image classification IEEE J Sel Top Appl Earth Observ Remote Sens 2017 10 5 1-14
[46]
Zhang Z, Zhao Y, Wang Y, et al. Transferring training instances for convenient cross-view object classification in surveillance IEEE Trans Inf Forens Sec 2013 8 10 1632-1641
[47]
Wu J, Zhu X, Zhang C, et al. Bag constrained structure pattern mining for multi-graph classification IEEE Trans Knowl Data Eng 2014 26 10 2382-2396
[48]
Wu J, Pan S, Zhu X, et al. Boosting for multi-graph classification IEEE Trans Cybern 2015 45 3 416-429
[49]
Dong Y, Liang T, Zhang Y, Du B (2020) Spectral-spatial weighted kernel manifold embedded distribution alignment for remote sensing image classification. IEEE Trans Cybern
[50]
Liang X, Lin L, Yang W, Luo P, Huang J, and Yan S Clothes co-parsing via joint image segmentation and labeling with application to clothing retrieval IEEE Trans Multimed 2016 18 6 1175-1186
[51]
Liu S, Feng J, Domokos C, Xu H, Huang J, Hu Z, and Yan S Fashion parsing with weak color-category labels IEEE Trans Multimed 2014 16 1 253-265
[52]
Elleuch M, Mezghani A, Khemakhem M et al (2019) Clothing classification using deep CNN architecture based on transfer learning. In: International conference on hybrid intelligent systems. Springer, Cham, pp 240–248
[53]
Wang Z, Du B, and Guo Y Domain adaptation with neural embedding matching IEEE Trans Neural Netw Learn Syst 2020 31 7 2387-2397
[54]
Oquab M, Bottou L, Laptev I, Sivic J (2014) Learning and transferring mid-level image representations using convolutional neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Columbus, OH, USA, pp 1717–1724
[55]
Liu S, Liang X, Liu L, Shen X, Yang J, Xu C, Lin L, Cao X, Yan S (2015) Matching-CNN meets KNN: quasi-parametric human parsing. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, USA, pp 1419–1427
[56]
Xu X, Gong S, Hospedales T (2013) Cross-domain traffic scene understanding by motion model transfer. In: Proceedings of the 4th ACM/IEEE international workshop on Analysis and retrieval of tracked events and motion in imagery stream (ARTEMIS), Barcelona, Spain, pp 77–86
[57]
Rajagopal AK, Subramanian R, Ricci E, Vieriu RL, Lanz O, Ramakrishnan KR, and Sebe N Exploring transfer learning approaches for head pose classification from multi-view surveillance images Int J Comput Vis 2014 109 1 146-167
[58]
Zheng WS (2012) Transfer re-identification: From person to set-based verification. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Providence, Rhode Island, pp 2650–2657
[59]
Li W, Zhao R, Wang X (2012) Human reidentification with transferred metric learning. In: Proceedings of the Asian conference on computer vision (ACCV), Daejeon, Korea, pp 31–44
[60]
Mckenna S Cross-scenario transfer person re-identification IEEE Trans Circuits Syst Video Technol 2015 26 8 1447-1460
[61]
Dong Q, Gong S, Zhu X (2017) Multi-task curriculum transfer deep learning of clothing attributes. In: IEEE winter conference on applications of computer vision (WACV), Santa Rosa, CA, USA, pp 520–529
[62]
Hoffman J, Tzeng E, Park T, Zhu JY, Isola P, Saenko K, Efros AA, Darrell T (2017) Cycada: Cycle-consistent adversarial domain adaptation. arXiv:1711.03213
[63]
Tsai YH, Hung WC, Schulter S, Sohn K, Yang MH, Chandraker M (2018) Learning to adapt structured output space for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Salt Lake City, Utah, USA
[64]
Kumar MP, Packer B, Koller D (2010) Self-paced learning for latent variable models. In: Proceedings of the international conference on neural information processing system (NIPS), British Columbia, Canada, pp 1189–1197
[65]
Jiang L, Meng D, Yu SI, Lan Z, Shan S, Hauptmann A (2014) Self-paced learning with diversity. Montreal, Quebec, Canada, pp 2078–2086
[66]
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), Boston, MA, USA, pp 3431–3440
[67]
Chatfield K, Simonyan K, Vedaldi A, Zisserman A (2014) Return of the devil in the details: delving deep into convolutional nets. In: Proceedings of the British machine vision conference (BMVC), Nottingham, UK
[68]
Luo P, Wang X, Tang X (2013) Pedestrian parsing via deep decompositional network. In: Proceedings of the IEEE international conference on computer vision (ICCV), Sydney, NSW, Australia, pp 2648–2655
[69]
Hirzer M, Beleznai C, Roth PM, Bischof H (2011) Person re-identification by descriptive and discriminative classification. In: Scandinavian conference on image analysis (SCIA), Ystad, Sweden, pp 91–102
[70]
Deng Y, Luo P, Loy CC, Tang X (2014) Pedestrian attribute recognition at far distance. In: Proceedings of the ACM international conference on multimedia (ACM MM), Orlando, FL, USA, pp 789–792
Recommendations
Transferring clothing parsing from fashion dataset to surveillance
2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)In this paper we address the problem of automatic clothing parsing in surveillance video with the information from user-generated tags such as “jeans” and “T-shirt”. Although clothing parsing has achieved great success in ...
Learning from Reduced Labels for Long-Tailed Data
ICMR '24: Proceedings of the 2024 International Conference on Multimedia RetrievalLong-tailed data is prevalent in real-world classification tasks and heavily relies on supervised information, which makes the annotation process exceptionally labor-intensive and time-consuming. Unfortunately, despite being a common approach to mitigate ...
Acknowledging the Unknown for Multi-label Learning with Single Positive Labels
Computer Vision – ECCV 2022AbstractDue to the difficulty of collecting exhaustive multi-label annotations, multi-label datasets often contain partial labels. We consider an extreme of this weakly supervised learning problem, called single positive multi-label learning (SPML), where ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
© Springer-Verlag London Ltd., part of Springer Nature 2020.
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 23 November 2020
Accepted: 10 November 2020
Received: 23 March 2020
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Natural Science Foundation of China
- National Natural Science Foundation of China
- National Natural Science Foundation of China
- National Natural Science Foundation of China
- National Natural Science Foundation of China
- National Key R&D Program of China
- Technology Research Program of Ministry of Public Security
- Hubei Province Technological Innovation Major Project
- Hubei Province Technological Innovation Major Project
- Nature Science Foundation of Hubei Province
- Nature Science Foundation of Jiangsu Province
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 20 Jan 2025