More Web Proxy on the site http://driver.im/

Article

Learning to Count in the Crowd from Limited Labeled Data

Authors:

Vishwanath A. Sindagi,

Rajeev Yasarla,

Deepak Sam Babu,

R. Venkatesh Babu,

Vishal M. PatelAuthors Info & Claims

Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI

Pages 212 - 229

https://doi.org/10.1007/978-3-030-58621-8_13

Published: 23 August 2020 Publication History

Abstract

Recent crowd counting approaches have achieved excellent performance. However, they are essentially based on fully supervised paradigm and require large number of annotated samples. Obtaining annotations is an expensive and labour-intensive process. In this work, we focus on reducing the annotation efforts by learning to count in the crowd from limited number of labeled samples while leveraging a large pool of unlabeled data. Specifically, we propose a Gaussian Process-based iterative learning mechanism that involves estimation of pseudo-ground truth for the unlabeled data, which is then used as supervision for training the network. The proposed method is shown to be effective under the reduced data (semi-supervised) settings for several datasets like ShanghaiTech, UCF-QNRF, WorldExpo, UCSD, etc. Furthermore, we demonstrate that the proposed method can be leveraged to enable the network in learning to count from synthetic dataset while being able to generalize better to real-world datasets (synthetic-to-real transfer).

References

[1]

Arteta C, Lempitsky V, and Zisserman A Leibe B, Matas J, Sebe N, and Welling M Counting in the wild Computer Vision 2016 Cham Springer 483-498

[2]

Babu Sam, D., Sajjan, N.N., Venkatesh Babu, R., Srinivasan, M.: Divide and grow: capturing huge diversity in crowd images with incrementally growing CNN. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3618–3626 (2018)

[3]

Cao X, Wang Z, Zhao Y, and Su F Ferrari V, Hebert M, Sminchisescu C, and Weiss Y Scale aggregation network for accurate and efficient crowd counting Computer Vision 2018 Cham Springer 757-773

[4]

Chan, A.B., Liang, Z.S.J., Vasconcelos, N.: Privacy preserving crowd monitoring: counting people without people models or tracking. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2008, pp. 1–7. IEEE (2008)

[5]

Chan, A.B., Vasconcelos, N.: Bayesian Poisson regression for crowd counting. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 545–551. IEEE (2009)

[6]

Change Loy, C., Gong, S., Xiang, T.: From semi-supervised to transfer counting of crowds. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2256–2263 (2013)

[7]

Chen, K., Loy, C.C., Gong, S., Xiang, T.: Feature mining for localised crowd counting. In: European Conference on Computer Vision (2012)

[8]

Gao, G., Gao, J., Liu, Q., Wang, Q., Wang, Y.: CNN-based density estimation and crowd counting: a survey. arXiv preprint arXiv:2003.12783 (2020)

[9]

Idrees, H., Saleemi, I., Seibert, C., Shah, M.: Multi-source multi-scale counting in extremely dense crowd images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2547–2554 (2013)

[10]

Idrees H et al. Ferrari V, Hebert M, Sminchisescu C, Weiss Y, et al. Composition loss for counting, density map estimation and localization in dense crowds Computer Vision 2018 Cham Springer 544-559

[11]

Jiang, X., et al.: Crowd counting and density estimation by trellis encoder-decoder network. arXiv preprint arXiv:1903.00853 (2019)

[12]

Kang, D., Ma, Z., Chan, A.B.: Beyond counting: comparisons of density maps for crowd analysis tasks-counting, detection, and tracking. arXiv preprint arXiv:1705.10118 (2017)

[13]

Laine, S., Aila, T.: Temporal ensembling for semi-supervised learning. arXiv preprint arXiv:1610.02242 (2016)

[14]

Lee, D.H.: Pseudo-label: the simple and efficient semi-supervised learning method for deep neural networks (2013)

[15]

Lempitsky, V., Zisserman, A.: Learning to count objects in images. In: Advances in Neural Information Processing Systems, pp. 1324–1332 (2010)

[16]

Li, M., Zhang, Z., Huang, K., Tan, T.: Estimating the number of people in crowded scenes by mid based foreground segmentation and head-shoulder detection. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)

[17]

Li T, Chang H, Wang M, Ni B, Hong R, and Yan S Crowded scene analysis: a survey IEEE Trans. Circ. Syst. Video Technol. 2015 25 3 367-386

[18]

Li W, Mahadevan V, and Vasconcelos N Anomaly detection and localization in crowded scenes IEEE Trans. Pattern Anal. Mach. Intell. 2014 36 1 18-32

[19]

Li, Y., Zhang, X., Chen, D.: CSRNet: dilated convolutional neural networks for understanding the highly congested scenes. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1091–1100 (2018)

[20]

Liu, N., Long, Y., Zou, C., Niu, Q., Pan, L., Wu, H.: ADCrowdNet: an attention-injective deformable convolutional network for crowd understanding. arXiv preprint arXiv:1811.11968 (2018)

[21]

Liu, W., Salzmann, M., Fua, P.: Context-aware crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5099–5108 (2019)

[22]

Liu, X., van de Weijer, J., Bagdanov, A.D.: Leveraging unlabeled data for crowd counting by learning to rank. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

[23]

Lu H, Cao Z, Xiao Y, Zhuang B, and Shen C TasselNet: counting maize tassels in the wild via local counts regression network Plant Methods 2017 13 1 79

[24]

Miyato T, Maeda SI, Koyama M, and Ishii S Virtual adversarial training: a regularization method for supervised and semi-supervised learning IEEE Trans. Pattern Anal. Mach. Intell. 2018 41 8 1979-1993

[25]

Oliver, A., Odena, A., Raffel, C.A., Cubuk, E.D., Goodfellow, I.: Realistic evaluation of deep semi-supervised learning algorithms. In: Advances in Neural Information Processing Systems, pp. 3235–3246 (2018)

[26]

Oñoro-Rubio D and López-Sastre RJ Leibe B, Matas J, Sebe N, and Welling M Towards perspective-free object counting with deep learning Computer Vision 2016 Cham Springer 615-629

[27]

Pham, V.Q., Kozakaya, T., Yamaguchi, O., Okada, R.: Count forest: co-voting uncertain number of targets using random forest for crowd density estimation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3253–3261 (2015)

[28]

Ranjan V, Le H, and Hoai M Ferrari V, Hebert M, Sminchisescu C, and Weiss Y Iterative crowd counting Computer Vision 2018 Cham Springer 278-293

[29]

Rasmussen CE Bousquet O, von Luxburg U, and Rätsch G Gaussian processes in machine learning Advanced Lectures on Machine Learning 2004 Heidelberg Springer 63-71

[30]

Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)

[31]

Ryan, D., Denman, S., Fookes, C., Sridharan, S.: Crowd counting using multiple local features. In: Digital Image Computing: Techniques and Applications, DICTA 2009, pp. 81–88. IEEE (2009)

[32]

Sam, D.B., Babu, R.V.: Top-down feedback for crowd counting convolutional neural network. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)

[33]

Sam, D.B., et al.: Locate, size and count: accurately resolving people in dense crowds via detection. arXiv preprint arXiv:1906.07538 (2019)

[34]

Sam, D.B., Peri, S.V., Mukuntha, N., Babu, R.V.: Going beyond the regression paradigm with accurate dot prediction for dense crowds. In: 2020 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2853–2861. IEEE (2020)

[35]

Sam, D.B., Sajjan, N.N., Maurya, H., Babu, R.V.: Almost unsupervised learning for dense crowd counting. In: Thirty-Third AAAI Conference on Artificial Intelligence (2019)

[36]

Sam, D.B., Surya, S., Babu, R.V.: Switching convolutional neural network for crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017)

[37]

Shen, Z., Xu, Y., Ni, B., Wang, M., Hu, J., Yang, X.: Crowd counting via adversarial cross-scale consistency pursuit. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

[38]

Shi, M., Yang, Z., Xu, C., Chen, Q.: Revisiting perspective information for efficient crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7279–7288 (2019)

[39]

Shi, Z., et al.: Crowd counting with deep negative correlation learning. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018

[40]

Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: International Conference on Learning Representations (2015)

[41]

Sindagi, V.A., Patel, V.M.: CNN-based cascaded multi-task learning of high-level prior and density estimation for crowd counting. In: 2017 IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS). IEEE (2017)

[42]

Sindagi, V.A., Patel, V.M.: Generating high-quality crowd density maps using contextual pyramid CNNs. In: The IEEE International Conference on Computer Vision (ICCV), October 2017

[43]

Sindagi VA and Patel VM A survey of recent advances in CNN-based single image crowd counting and density estimation Pattern Recogn. Lett. 2017 107 3-16

[44]

Sindagi, V.A., Patel, V.M.: DAFE-FD: Density aware feature enrichment for face detection. In: 2019 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 2185–2195. IEEE (2019)

[45]

Sindagi, V.A., Patel, V.M.: HA-CCN: Hierarchical attention-based crowd counting network. arXiv preprint arXiv:1907.10255 (2019)

[46]

Sindagi, V.A., Patel, V.M.: Inverse attention guided deep crowd counting network. arXiv preprint (2019)

[47]

Sindagi, V.A., Patel, V.M.: Multi-level bottom-top and top-bottom feature fusion for crowd counting. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1002–1012 (2019)

[48]

Sindagi, V.A., Yasarla, R., Patel, V.M.: Pushing the frontiers of unconstrained crowd counting: new dataset and benchmark method. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1221–1231 (2019)

[49]

Sindagi, V.A., Yasarla, R., Patel, V.M.: JHU-CROWD++: large-scale crowd counting dataset and a benchmark method. arXiv preprint arXiv:2004.03597 (2020)

[50]

Toropov, E., Gui, L., Zhang, S., Kottur, S., Moura, J.M.: Traffic flow from a low frame rate city camera. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3802–3806. IEEE (2015)

[51]

Walach E and Wolf L Leibe B, Matas J, Sebe N, and Welling M Learning to count with CNN boosting Computer Vision 2016 Cham Springer 660-676

[52]

Wan, J., Chan, A.: Adaptive density map generation for crowd counting. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1130–1139 (2019)

[53]

Wan, J., Luo, W., Wu, B., Chan, A.B., Liu, W.: Residual regression with semantic prior for crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4036–4045 (2019)

[54]

Wang, C., Zhang, H., Yang, L., Liu, S., Cao, X.: Deep people counting in extremely dense crowds. In: Proceedings of the 23rd ACM international conference on Multimedia, pp. 1299–1302. ACM (2015)

[55]

Wang, Q., Gao, J., Lin, W., Yuan, Y.: Learning from synthetic data for crowd counting in the wild. arXiv preprint arXiv:1903.03303 (2019)

[56]

Xu, B., Qiu, G.: Crowd density estimation based on rich features and random projection forest. In: 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–8. IEEE (2016)

[57]

Yasarla, R., Sindagi, V.A., Patel, V.M.: Syn2Real transfer learning for image deraining using Gaussian processes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2726–2736 (2020)

[58]

Zhan B, Monekosso DN, Remagnino P, Velastin SA, and Xu LQ Crowd analysis: a survey Mach. Vis. Appl. 2008 19 5–6 345-357

[59]

Zhang, C., Li, H., Wang, X., Yang, X.: Cross-scene crowd counting via deep convolutional neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 833–841 (2015)

[60]

Zhang, Q., Chan, A.B.: Wide-area crowd counting via ground-plane density maps and multi-view fusion CNNs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8297–8306 (2019)

[61]

Zhang, Y., Zhou, D., Chen, S., Gao, S., Ma, Y.: Single-image crowd counting via multi-column convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 589–597 (2016)

[62]

Zhao, M., Zhang, J., Zhang, C., Zhang, W.: Leveraging heterogeneous auxiliary tasks to assist crowd counting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 12736–12745 (2019)

[63]

Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

Cited By

Cunha WMoreo Fernández AEsuli ASebastiani FRocha LGonçalves M(2025)A Noise-Oriented and Redundancy-Aware Instance Selection FrameworkACM Transactions on Information Systems10.1145/370500043:2(1-33)Online publication date: 17-Jan-2025
https://dl.acm.org/doi/10.1145/3705000
Katyal JPoullis C(2023)Strategic Incorporation of Synthetic Data for Performance Enhancement in Deep Learning A Case Study on Object Tracking TasksAdvances in Visual Computing10.1007/978-3-031-47969-4_40(513-528)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-47969-4_40
Yang KHuang YHuang JHsu YWan CShuai HWang LCheng W(2022)Improving Crowd Density Estimation by Fusing Aerial Images and Radio SignalsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/349234618:3(1-23)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3492346
Show More Cited By

Index Terms

Learning to Count in the Crowd from Limited Labeled Data
1. Computing methodologies
  1. Artificial intelligence
  2. Machine learning
2. Information systems
  1. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Semi-supervised learning using multiple clusterings with limited labeled data

Supervised classification consists in learning a predictive model using a set of labeled samples. It is accepted that predictive models accuracy usually increases as more labeled samples are available. Labeled samples are generally difficult to obtain ...
Non-linear dictionary learning with partially labeled data

While recent techniques for discriminative dictionary learning have demonstrated tremendous success in image analysis applications, their performance is often limited by the amount of labeled data available for training. Even though labeling images is ...
Learning Instance Weighted Naive Bayes from labeled and unlabeled data

In real-world data mining applications, it is often the case that unlabeled instances are abundant, while available labeled instances are very limited. Thus, semi-supervised learning, which attempts to benefit from large amount of unlabeled data ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Guide Proceedings

Computer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XI

Aug 2020

857 pages

ISBN:978-3-030-58620-1

DOI:10.1007/978-3-030-58621-8

Editors:
Andrea Vedaldi
University of Oxford, Oxford, UK
,
Horst Bischof
Graz University of Technology, Graz, Austria
,
Thomas Brox
University of Freiburg, Freiburg im Breisgau, Germany
,
Jan-Michael Frahm
University of North Carolina at Chapel Hill, Chapel Hill, NC, USA

© Springer Nature Switzerland AG 2020.

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 23 August 2020

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cunha WMoreo Fernández AEsuli ASebastiani FRocha LGonçalves M(2025)A Noise-Oriented and Redundancy-Aware Instance Selection FrameworkACM Transactions on Information Systems10.1145/370500043:2(1-33)Online publication date: 17-Jan-2025
https://dl.acm.org/doi/10.1145/3705000
Katyal JPoullis C(2023)Strategic Incorporation of Synthetic Data for Performance Enhancement in Deep Learning A Case Study on Object Tracking TasksAdvances in Visual Computing10.1007/978-3-031-47969-4_40(513-528)Online publication date: 16-Oct-2023
https://dl.acm.org/doi/10.1007/978-3-031-47969-4_40
Yang KHuang YHuang JHsu YWan CShuai HWang LCheng W(2022)Improving Crowd Density Estimation by Fusing Aerial Images and Radio SignalsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/349234618:3(1-23)Online publication date: 4-Mar-2022
https://dl.acm.org/doi/10.1145/3492346
Hu YJia RLiu YLi YSun H(2022)WSNetKnowledge-Based Systems10.1016/j.knosys.2022.109727255:COnline publication date: 14-Nov-2022
https://dl.acm.org/doi/10.1016/j.knosys.2022.109727

View Options

View options

Figures

Tables

Media

View Table of Conten