Image Segmentation based Background Removal and Replacement
Authors: 
Meenal Gupta, Ritik Goyal, Siddharth Shekhar, Rajalakshmi KrishnamurthiAuthors Info & Claims
Pages 77 - 85
Abstract
Privacy is a great cause of concern. As many of us are concerned and self-conscious about what is visible in our background while having an online meeting. To provide some privacy during an online meeting our main objective is to blur/remove/replace the background of a live webcam video stream using a machine learning model trained to segment out persons in an image, whose results are then refined using image processing techniques. Currently, it is available in online meeting apps like Zoom, Google Meet, Microsoft Teams, etc., but there is no open-source code available for it so we also want to make an online repository, from where anyone can take our code, and implement it in their project. Our program uses a webcam on a computer and proceeds in the following way. First, we capture a live frame from the webcam, and then pre-process it, so that it is suitable for supplying as input to the machine-learning model. This provides us with a rough predicted foreground mask. Then, to refine it, we perform morphological closing, which is used to remove small imperfections in the mask. We then obtain the outline of the detected person using contour finding and fill its interior region with white by using a flood-fill operation. After all, of this, we have generated a foreground mask layer in an alpha channel whose white area shows the foreground, which we have to leave as it, is and the black area shows background, which we have to change. Therefore, we combine this generated mask along with its corresponding frame and, optionally, a background image, to generate its corresponding frame. Repeatedly doing this for every frame we receive from the webcam continuously, we have a live video with edited background with a decent frame rate of 26-28 FPS.
References
[1]
LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11), 2278–2323. https://doi.org/10.1109/5.726791
[2]
Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2017). ImageNet classification with deep convolutional neural networks. Communications of the ACM, 60(6), 84-90.
[3]
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December, 770–778. https://doi.org/10.1109/CVPR.2016.90
[4]
Simonyan, K., & Zisserman, A. (2015). Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 1–14.
[5]
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., ... & Rabinovich, A. (2015). Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1-9).
[6]
Girshick, R., Donahue, J., Darrell, T., & Malik, J. (2016). Region-Based Convolutional Networks for Accurate Object Detection and Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 38(1), 142–158. https://doi.org/10.1109/TPAMI.2015.2437384
[7]
Girshick, R. (2015). Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision, 2015 International Conference on Computer Vision, ICCV 2015, 1440–1448. https://doi.org/10.1109/ICCV.2015.169
[8]
Ren, S., He, K., Girshick, R., & Sun, J. (2017). Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(6), 1137–1149. https://doi.org/10.1109/TPAMI.2016.2577031
[9]
He, K., Gkioxari, G., Dollar, P., & Girshick, R. (2017). Mask R-CNN. Proceedings of the IEEE International Conference on Computer Vision, 2017-October, 2980–2988. https://doi.org/10.1109/ICCV.2017.322
[10]
Krishnamurthi, R., Maheshwari, R., & Gulati, R. (2020). Colorizing and Captioning Images Using Deep Learning Models and Deploying Them Via IoT Deployment Tools. International Journal of Information Retrieval Research (IJIRR), 10(4), 35-50.
Recommendations
Image Segmentation Using Proportion of Foreground to Background Algorithm
IS3C '12: Proceedings of the 2012 International Symposium on Computer, Consumer and ControlThe medical image is widely used in medicine diagnosis for the science-sound qualitative and quantitative analysis. To achieve a clear image, the technology of the image segmentation is the key point. Therefore, this paper proposes a new image ...
Multiscale background modelling and segmentation
DSP'09: Proceedings of the 16th international conference on Digital Signal ProcessingA new multiscale approach to motion based segmentation of objects in video sequences is presented. While image features extracted at multiple scales are commonly used within the pattern recognition community, they have seldom been employed for ...
Hand Gesture Detection and Segmentation Based on Difference Background Image with Complex Background
ICESS '08: Proceedings of the 2008 International Conference on Embedded Software and SystemsThe video streams of backgrounds are frequently influenced by the background changes (e.g. illumination changes and changes due to adding or removing parts of the background). Further more, the quality of the foreground and the segmented image of hand ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
August 2021
483 pages
ISBN:9781450389204
DOI:10.1145/3474124
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 04 November 2021
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
IC3 '21
IC3 '21: 2021 Thirteenth International Conference on Contemporary Computing
August 5 - 7, 2021
Noida, India
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 90Total Downloads
- Downloads (Last 12 months)17
- Downloads (Last 6 weeks)0
Reflects downloads up to 07 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format