[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3394171.3413972acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Privacy-sensitive Objects Pixelation for Live Video Streaming

Published: 12 October 2020 Publication History

Abstract

With the prevailing of live video streaming, establishing an online pixelation method for privacy-sensitive objects is an urgency. Caused by the inaccurate detection of privacy-sensitive objects, simply migrating the tracking-by-detection structure applied in offline pixelation into the online form will incur problems in target initialization, drifting, and over-pixelation. To cope with the inevitable but impacting detection issue, we propose a novel Privacy-sensitive Objects Pixelation (PsOP) framework for automatic personal privacy filtering during live video streaming. Leveraging pre-trained detection networks, our PsOP is extendable to any potential privacy-sensitive objects pixelation. Employing the embedding networks and the proposed Positioned Incremental Affinity Propagation (PIAP) clustering algorithm as the backbone, our PsOP unifies the pixelation of discriminating and indiscriminating pixelation objects through trajectories generation. In addition to the pixelation accuracy boosting, experiment results on the streaming video data we built show that the proposed PsOP can significantly reduce the over-pixelation ratio in privacy-sensitive object pixelation.

Supplementary Material

ZIP File (mmfp0442aux.zip)
This is the Supplementary Material for paper "Privacy-sensitive Objects Pixelation for Live Video Streaming" published in ACM MM 2020. It contains 2 files: The .pdf file is a text document seals image results and corresponding descriptions for quantitative analysis to support the experiments in the papers.
MP4 File (3394171.3413972.mp4)
We demonstrate a video sample in this presentation to explicitly exhibit the advantages of PsOP comparing with applied, tracking-based, offline pixelation tools. The proposed PsOP manages to solve the pixelation problems in video live streaming through the proposed Positioned Incremental Affinity Propagation (PIAP) clustering and pre-trained CNNs. PsOP unifies the pixelation of discriminating and indiscriminating pixelation objects through trajectories generation. PsOP shows significant improvements in pixelation accuracy, precision, and over-pixelation ratio compared with other offline pixelation algorithms.

References

[1]
Sandra Avila, Nicolas Thome, Matthieu Cord, Eduardo Valle, and Arnaldo De A AraúJo. 2013. Pooling in image representation: The visual codeword point of view. Computer Vision and Image Understanding 117, 5 (2013), 453--465.
[2]
Jerome Berclaz, Francois Fleuret, Engin Turetken, and Pascal Fua. 2011. Multiple object tracking using k-shortest paths optimization. IEEE transactions on pattern analysis and machine intelligence 33, 9 (2011), 1806--1819.
[3]
YouTube Official Blog. January 6, 2020 (accessed Apr 17th, 2020). Better protecting kids? privacy on YouTube. https://youtube.googleblog.com/2020/01/betterprotecting- kids-privacy-on-YouTube.html
[4]
Danushka Bollegala, David Weir, and John Carroll. 2012. Cross-domain sentiment classification using a sentiment sensitive thesaurus. IEEE transactions on knowledge and data engineering 25, 8 (2012), 1719--1731.
[5]
Martin Danelljan, Goutam Bhat, Fahad Shahbaz Khan, and Michael Felsberg. 2017. Eco: Efficient convolution operators for tracking. In Proceedings of the IEEE conference on computer vision and pattern recognition. 6638--6646.
[6]
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. 2019. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4690--4699.
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[8]
Martin Ester, Hans-Peter Kriegel, Jörg Sander, and Xiaowei Xu. 1996. Densitybased spatial clustering of applications with noise. In Int. Conf. Knowledge Discovery and Data Mining, Vol. 240. 6.
[9]
Cori Faklaris, Francesco Cafaro, Sara Anne Hook, Asa Blevins, Matt O'Haver, and Neha Singhal. 2016. Legal and ethical implications of mobile live-streaming video apps. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. ACM, 722--729.
[10]
Brendan J Frey and Delbert Dueck. 2007. Clustering by passing messages between data points. science 315, 5814 (2007), 972--976.
[11]
Guojun Gan and Michael Kwok-Po Ng. 2015. Subspace clustering using affinity propagation. Pattern Recognition 48, 4 (2015), 1455--1464.
[12]
YouTube Help. 2020 (accessed Apr 17th, 2020). Blur your videos. https://support. google.com/youtube/answer/9057652?hl=en
[13]
João F Henriques, Rui Caseiro, Pedro Martins, and Jorge Batista. 2014. High-speed tracking with kernelized correlation filters. IEEE transactions on pattern analysis and machine intelligence 37, 3 (2014), 583--596.
[14]
Shuiwang Ji, Wei Xu, Ming Yang, and Kai Yu. 2013. 3D convolutional neural networks for human action recognition. IEEE transactions on pattern analysis and machine intelligence 35, 1 (2013), 221--231.
[15]
Armand Joulin, Edouard Grave, Piotr Bojanowski, Matthijs Douze, Hérve Jégou, and Tomas Mikolov. 2016. Fasttext. zip: Compressing text classification models. arXiv preprint arXiv:1612.03651 (2016).
[16]
Juliako. 2020 (accessed Apr 17th, 2020). Redact faces with Azure Media Analytics. https://docs.microsoft.com/en-us/azure/media-services/previous/mediaservices- face-redaction
[17]
Andrej Karpathy, George Toderici, Sanketh Shetty, Thomas Leung, Rahul Sukthankar, and Li Fei-Fei. 2014. Large-scale video classification with convolutional neural networks. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition. 1725--1732.
[18]
Feng Li, Cheng Tian, Wangmeng Zuo, Lei Zhang, and Ming-Hsuan Yang. 2018. Learning spatial-temporal regularized correlation filters for visual tracking. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4904--4913.
[19]
Minghui Liao, Baoguang Shi, Xiang Bai, Xinggang Wang, and Wenyu Liu. 2017. Textboxes: A fast text detector with a single deep neural network. In Thirty-First AAAI Conference on Artificial Intelligence.
[20]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579--2605.
[21]
Nikhil R Pal and James C Bezdek. 1995. On cluster validity for the fuzzy c-means model. IEEE Transactions on Fuzzy systems 3, 3 (1995), 370--379.
[22]
Ray Smith. 2007. An overview of the Tesseract OCR engine. In Ninth international conference on document analysis and recognition (ICDAR 2007), Vol. 2. IEEE, 629-- 633.
[23]
Daxton Râ Chipâ Stewart and Jeremy Littau. 2016. Up, periscope: Mobile streaming video technologies, privacy in public, and the right to record. Journalism & Mass Communication Quarterly 93, 2 (2016), 312--331.
[24]
Leilei Sun and Chonghui Guo. 2014. Incremental affinity propagation clustering based on message passing. IEEE Transactions on Knowledge and Data Engineering 26, 11 (2014), 2731--2744.
[25]
Xu Tang, Daniel K Du, Zeqiang He, and Jingtuo Liu. 2018. Pyramidbox: A contextassisted single shot face detector. In Proceedings of the European Conference on Computer Vision (ECCV). 797--813.
[26]
Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. 2015. Learning spatiotemporal features with 3d convolutional networks. In Proceedings of the IEEE international conference on computer vision. 4489--4497.
[27]
Osman Tursun, Cemal Aker, and Sinan Kalkan. 2017. A large-scale dataset and benchmark for similar trademark retrieval. arXiv preprint arXiv:1701.05766 (2017).
[28]
Chang-Dong Wang, Jian-Huang Lai, Ching Y Suen, and Jun-Yong Zhu. 2013. Multi-exemplar affinity propagation. IEEE transactions on pattern analysis and machine intelligence 35, 9 (2013), 2223--2237.
[29]
Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. 2018. Cosface: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5265--5274.
[30]
Fengwei Yu, Wenbo Li, Quanquan Li, Yu Liu, Xiaohua Shi, and Junjie Yan. 2016. Poi: Multiple object tracking with high performance detection and appearance feature. In European Conference on Computer Vision. Springer, 36--42.
[31]
Jun Yu, Zhenzhong Kuang, Zhou Yu, Dan Lin, and Jianping Fan. 2017. Privacy setting recommendation for image sharing. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). IEEE, 726--730.
[32]
Jun Yu, Baopeng Zhang, Zhengzhong Kuang, Dan Lin, and Jianping Fan. 2016. iPrivacy: image privacy protection by identifying sensitive objects via deep multitask learning. IEEE Transactions on Information Forensics and Security 12, 5 (2016), 1005--1016.
[33]
Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga, and George Toderici. 2015. Beyond short snippets: Deep networks for video classification. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4694--4702.
[34]
Amir Roshan Zamir, Afshin Dehghan, and Mubarak Shah. 2012. Gmcp-tracker: Global multi-object tracking using generalized minimum clique graphs. In European Conference on Computer Vision. Springer, 343--356.
[35]
Kaipeng Zhang, Zhanpeng Zhang, Zhifeng Li, and Yu Qiao. 2016. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters 23, 10 (2016), 1499--1503.
[36]
Tong Zhang and Herman Martins Gomes. 2014. Technology survey on video face tracking. In Imaging and Multimedia Analytics in a Web and Mobile World 2014, Vol. 9027. International Society for Optics and Photonics, 90270F.
[37]
Zheng Zhu, Wei Wu, Wei Zou, and Junjie Yan. 2018. End-to-end flow correlation tracking with spatial-temporal attention. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 548--557.

Cited By

View all
  • (2024)Deep Motion Flow Guided Reversible Face Video De-identificationFace De-identification: Safeguarding Identities in the Digital Era10.1007/978-3-031-58222-6_8(147-176)Online publication date: 26-Apr-2024
  • (2023)Enhancing Free-Living Fall Risk Assessment: Contextualizing Mobility Based IMU DataSensors10.3390/s2302089123:2(891)Online publication date: 12-Jan-2023
  • (2023)Disability-First Design and Creation of A Dataset Showing Private Visual Information Collected With People Who Are BlindProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580922(1-15)Online publication date: 19-Apr-2023
  • Show More Cited By

Index Terms

  1. Privacy-sensitive Objects Pixelation for Live Video Streaming

    Recommendations

    Comments

    Please enable JavaScript to view thecomments powered by Disqus.

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '20: Proceedings of the 28th ACM International Conference on Multimedia
    October 2020
    4889 pages
    ISBN:9781450379885
    DOI:10.1145/3394171
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 October 2020

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. objects pixelation
    2. privacy protection

    Qualifiers

    • Research-article

    Conference

    MM '20
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)77
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 12 Dec 2024

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Deep Motion Flow Guided Reversible Face Video De-identificationFace De-identification: Safeguarding Identities in the Digital Era10.1007/978-3-031-58222-6_8(147-176)Online publication date: 26-Apr-2024
    • (2023)Enhancing Free-Living Fall Risk Assessment: Contextualizing Mobility Based IMU DataSensors10.3390/s2302089123:2(891)Online publication date: 12-Jan-2023
    • (2023)Disability-First Design and Creation of A Dataset Showing Private Visual Information Collected With People Who Are BlindProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580922(1-15)Online publication date: 19-Apr-2023
    • (2023)Pre-training-free Image Manipulation Localization through Non-Mutually Exclusive Contrastive Learning2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02042(22289-22299)Online publication date: 1-Oct-2023
    • (2022)IdentityMask: Deep Motion Flow Guided Reversible Face Video De-IdentificationIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2022.319198232:12(8353-8367)Online publication date: Dec-2022
    • (2022)Global Optimization Solution for Dynamic Adaptive 360-Degree StreamingICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP43922.2022.9746184(1-5)Online publication date: 23-May-2022
    • (2021)Deep Motion Flow Aided Face Video De-identification2021 International Conference on Visual Communications and Image Processing (VCIP)10.1109/VCIP53242.2021.9675353(1-5)Online publication date: 5-Dec-2021

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media