More Web Proxy on the site http://driver.im/

research-article

Enhanced Reweighted MRFs for Efficient Fashion Image Parsing

Authors:

Pierre BoulangerAuthors Info & Claims

ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Volume 12, Issue 3

Article No.: 42, Pages 1 - 16

https://doi.org/10.1145/2890104

Published: 08 March 2016 Publication History

Abstract

Previous image parsing methods usually model the problem in a conditional random field which describes a statistical model learned from a training dataset and then processes a query image using the conditional probability. However, for clothing images, fashion items have a large variety of layering and configuration, and it is hard to learn a certain statistical model of features that apply to general cases. In this article, we take fashion images as an example to show how Markov Random Fields (MRFs) can outperform Conditional Random Fields when the application does not follow a certain statistical model learned from the training data set. We propose a new method for automatically parsing fashion images in high processing efficiency with significantly less training time by applying a modification of MRFs, named reweighted MRF (RW-MRF), which resolves the problem of over smoothing infrequent labels. We further enhance RW-MRF with occlusion prior and background prior to resolve two other common problems in clothing parsing, occlusion, and background spill. Our experimental results indicate that our proposed clothing parsing method significantly improves processing time and training time over state-of-the-art methods, while ensuring comparable parsing accuracy and improving label recall rate.

Supplementary Material

wu (wu.zip)

Supplemental movie, appendix, image and software files for, Enhanced Reweighted MRFs for Efficient Fashion Image Parsing

Download
26.48 MB

References

[1]

Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Susstrunk. 2012. SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans. Pattern Anal. Mach. Intell. 34, 11 (Nov 2012), 2274--2282.

Digital Library

[2]

Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik. 2011. Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 5 (May 2011), 898--916.

Digital Library

[3]

Yuri Boykov and Vladimir Kolmogorov. 2004. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26, 9 (Sept. 2004), 1124--1137.

Digital Library

[4]

J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: A large-scale hierarchical image database. In CVPR09.

[5]

Jian Dong, Qiang Chen, Xiaohui Shen, Jianchao Yang, and Shuicheng Yan. 2014. Towards unified human parsing and pose estimation. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14).IEEE, Washington, DC, 843--850.

Digital Library

[6]

Jian Dong, Qiang Chen, Wei Xia, Zhongyang Huang, and Shuicheng Yan. 2013. A deformable mixture parsing model with parselets. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV’13). IEEE Computer Society, Washington, DC, 3408--3415.

Digital Library

[7]

Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A library for large linear classification. J. Mach. Learn. Res. 9 (June 2008), 1871--1874.

Digital Library

[8]

Li Fei-Fei, R. Fergus, and P. Perona. 2006. One-shot learning of object categories. IEEE Trans. Pattern Anal. Machine Intell. 28, 4 (April 2006), 594--611.

Digital Library

[9]

Basela Hasan and David Hogg. 2010. Segmentation using deformable spatial priors with application to clothing. In Proceedings of the British Machine Vision Conference. BMVA Press, 83.1--83.11.

[10]

Xuming He, Richard S. Zemel, and Miguel Á. Carreira-Perpiñán. 2004. Multiscale conditional random fields for image labeling. In Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004 (CVPR 2004), Vol. 2. IEEE, Washington, DC, II--695.

Digital Library

[11]

Yannis Kalantidis, Lyndon Kennedy, and Li-Jia Li. 2013. Getting the Look: Clothing recognition and segmentation for automatic product suggestions in everyday photos. In Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval (ICMR’13). ACM, New York, NY, 105--112.

Digital Library

[12]

E. Kim, XiaoLei Huang, and Gang Tan. 2011. Markup SVG: An online content-aware image abstraction and annotation tool. IEEE Trans. Multimed. 13, 5 (Oct 2011), 993--1006.

Digital Library

[13]

Thomas Leung and Jitendra Malik. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vision 43, 1 (2001), 29--44.

Digital Library

[14]

Xiaodan Liang, Si Liu, Xiaohui Shen, Jianchao Yang, Luoqi Liu, Jian Dong, Liang Lin, and Shuicheng Yan. 2015a. Deep human parsing with active template regression. IEEE Transactions on Pattern Analysis and Machine Intelligence 37, 12 (2015), 2402--2414.

Digital Library

[15]

Xiaodan Liang, Chunyan Xu, Xiaohui Shen, Jianchao Yang, Si Liu, Jinhui Tang, Liang Lin, and Shuicheng Yan. 2015b. Human parsing with contextualized convolutional neural network. In Proceedings of the IEEE International Conference on Computer Vision. 1386--1394.

Digital Library

[16]

Si Liu, Jiashi Feng, C. Domokos, Hui Xu, Junshi Huang, Zhenzhen Hu, and Shuicheng Yan. 2014. Fashion parsing with weak color-category labels. IEEE Trans. Multimed. 16, 1 (Jan 2014), 253--265.

[17]

Si Liu, Xiaodan Liang, Luoqi Liu, Xiaohui Shen, Jianchao Yang, Changsheng Xu, Liang Lin, Xiaochun Cao, and Shuicheng Yan. 2015. Matching-cnn meets KNN: Quasi-parametric human parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1419--1427.

[18]

Bryan C. Russell, Antonio Torralba, Kevin P. Murphy, and William T. Freeman. 2008. LabelMe: A database and web-based tool for image annotation. Int. J. Comput. Vision 77, 1--3 (May 2008), 157--173.

Digital Library

[19]

János Schanda. 2007. Colorimetry: Understanding the CIE System. John Wiley & Sons, New York, NY.

[20]

Jamie Shotton, John Winn, Carsten Rother, and Antonio Criminisi. 2006. TextonBoost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In Proceedings of the 9th European Conference on Computer Vision—Volume Part I (ECCV’06). Springer-Verlag, Berlin, 1--15.

Digital Library

[21]

Joseph Tighe and Svetlana Lazebnik. 2010. Superparsing: Scalable nonparametric image parsing with superpixels. In Computer Vision—ECCV 2010. Springer, Berlin, 352--365.

Digital Library

[22]

Zhuowen Tu, Xiangrong Chen, Alan L. Yuille, and Song-Chun Zhu. 2005. Image parsing: Unifying segmentation, detection, and recognition. Int. J. Comput. Vision 63, 2 (2005), 113--140.

Digital Library

[23]

Yichen Wei, Fang Wen, Wangjiang Zhu, and Jian Sun. 2012. Geodesic saliency using background priors. In Proceedings of the 12th European Conference on Computer Vision—Volume Part III (ECCV’12). Springer-Verlag, Berlin, 29--42.

Digital Library

[24]

John Winn and Nebojsa Jojic. 2005. Locus: Learning object classes with unsupervised segmentation. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV’05). Vol. 1. IEEE, Washigton, DC, 756--763.

Digital Library

[25]

Qiong Wu, Rui Gao, Xida Chen, and Pierre Boulanger. 2014. Tagging driven by interactive image discovery: Tagging-tracking-learning. In Proceedings of the 2014 IEEE International Symposium on Multimedia (ISM’14). IEEE, Washington, DC, 179--186.

Digital Library

[26]

K. Yamaguchi, M. H. Kiapour, and T. L. Berg. 2013. Paper doll parsing: Retrieving similar styles to parse clothing items. In Proceedings of the 2013 IEEE International Conference on Computer Vision (ICCV’13). IEEE, Washington, DC, 3519--3526.

Digital Library

[27]

Kota Yamaguchi, M. Hadi Kiapour, Luis E. Ortiz, and Tamara L. Berg. 2012. Parsing clothing in fashion photographs. In Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’12). IEEE, Washington, DC, 3570--3577.

Digital Library

[28]

Wei Yang, Ping Luo, and Liang Lin. 2014. Clothing co-parsing by joint image segmentation and labeling. In Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’14). IEEE, Washington, DC, 3182--3189.

Digital Library

[29]

Yi Yang and D. Ramanan. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR’11). IEEE Computer Society, Washington, DC, 1385--1392.

Digital Library

[30]

Shuai Zheng, Sadeep Jayasumana, Bernardino Romera-Paredes, Vibhav Vineet, Zhizhong Su, Dalong Du, Chang Huang, and Philip Torr. 2015. Conditional random fields as recurrent neural networks. arXiv:1502.03240 (2015).

Cited By

Yu FZhang YLi HDu CLiu LJiang M(2024)Phase Contour Enhancement Network for Clothing ParsingIEEE Transactions on Consumer Electronics10.1109/TCE.2024.337737770:1(2784-2793)Online publication date: 19-Mar-2024
https://dl.acm.org/doi/10.1109/TCE.2024.3377377
Dong JHuo QFerrari S(2022)A Holistic Approach for Role Inference and Action Anticipation in Human TeamsACM Transactions on Intelligent Systems and Technology10.1145/353123013:6(1-24)Online publication date: 22-Sep-2022
https://dl.acm.org/doi/10.1145/3531230
Ye SWang SFan JXu AMa XShi X(2022)Dual Context Based Network for Clothing Parsing2022 7th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA)10.1109/ICCCBDA55098.2022.9778929(453-457)Online publication date: 22-Apr-2022
https://doi.org/10.1109/ICCCBDA55098.2022.9778929
Show More Cited By

Index Terms

Enhanced Reweighted MRFs for Efficient Fashion Image Parsing
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Image segmentation
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interactive systems and tools

Recommendations

Learning CRFs for Image Parsing with Adaptive Subgradient Descent
ICCV '13: Proceedings of the 2013 IEEE International Conference on Computer Vision

We propose an adaptive sub gradient descent method to efficiently learn the parameters of CRF models for image parsing. To balance the learning efficiency and performance of the learned CRF models, the parameter learning is iteratively carried out by ...
An HMM/MRF-based stochastic framework for robust vehicle tracking

Shadows of moving objects often obstruct robust visual tracking. In this paper, we present a car tracker based on a hidden Markov model/Markov random field (HMM/MRF)-based segmentation method that is capable of classifying each small region of an image ...
Image Segmentation with a Unified Graphical Model

We propose a unified graphical model that can represent both the causal and noncausal relationships among random variables and apply it to the image segmentation problem. Specifically, we first propose to employ Conditional Random Field (CRF) to model ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Multimedia Computing, Communications, and Applications

ACM Transactions on Multimedia Computing, Communications, and Applications Volume 12, Issue 3

June 2016

227 pages

ISSN:1551-6857

EISSN:1551-6865

DOI:10.1145/2901366

Editor:
Alberto Del Bimbo
University of Firenze, Italy

Issue’s Table of Contents

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 March 2016

Revised: 01 November 2015

Accepted: 01 September 2015

Received: 01 August 2015

Published in TOMM Volume 12, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

20
Total Citations
View Citations
231
Total Downloads

Downloads (Last 12 months)5
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yu FZhang YLi HDu CLiu LJiang M(2024)Phase Contour Enhancement Network for Clothing ParsingIEEE Transactions on Consumer Electronics10.1109/TCE.2024.337737770:1(2784-2793)Online publication date: 19-Mar-2024
https://dl.acm.org/doi/10.1109/TCE.2024.3377377
Dong JHuo QFerrari S(2022)A Holistic Approach for Role Inference and Action Anticipation in Human TeamsACM Transactions on Intelligent Systems and Technology10.1145/353123013:6(1-24)Online publication date: 22-Sep-2022
https://dl.acm.org/doi/10.1145/3531230
Ye SWang SFan JXu AMa XShi X(2022)Dual Context Based Network for Clothing Parsing2022 7th International Conference on Cloud Computing and Big Data Analytics (ICCCBDA)10.1109/ICCCBDA55098.2022.9778929(453-457)Online publication date: 22-Apr-2022
https://doi.org/10.1109/ICCCBDA55098.2022.9778929
Zhang DZuo CWu QFu LXiang X(2022)Unabridged adjacent modulation for clothing parsingPattern Recognition10.1016/j.patcog.2022.108594127:COnline publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1016/j.patcog.2022.108594
Su ZChen MHuang ELin GZhou F(2021)MVSNNeurocomputing10.1016/j.neucom.2021.08.124465:C(437-450)Online publication date: 20-Nov-2021
https://dl.acm.org/doi/10.1016/j.neucom.2021.08.124
Jr. GTristan J(2019)Using Butterfly-patterned Partial Sums to Draw from Discrete DistributionsACM Transactions on Parallel Computing10.1145/33656626:4(1-30)Online publication date: 19-Nov-2019
https://dl.acm.org/doi/10.1145/3365662
Anjum B(2019)An interview with Lana YaroshUbiquity10.1145/33386282019:June(1-7)Online publication date: 11-Jun-2019
https://dl.acm.org/doi/10.1145/3338628
Odlyzko A(2019)Cybersecurity is not very importantUbiquity10.1145/33336112019:June(1-23)Online publication date: 14-Jun-2019
https://dl.acm.org/doi/10.1145/3333611
Feng ZYu ZJing YWu SSong MYang YJiang J(2019)Interpretable Partitioned Embedding for Intelligent Multi-item Fashion Outfit CompositionACM Transactions on Multimedia Computing, Communications, and Applications10.1145/332633215:2s(1-20)Online publication date: 29-Jul-2019
https://doi.org/10.1145/3326332
Lim BRogers YSebire N(2019)Designing to DistractACM Transactions on Computer-Human Interaction10.1145/330142726:2(1-19)Online publication date: 9-Apr-2019
https://dl.acm.org/doi/10.1145/3301427
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents