[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
10.1145/3552467.3554798acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article
Open access

Impact of Blind Image Quality Assessment on the Retrieval of Lifelog Images

Published: 10 October 2022 Publication History

Abstract

The use of personal lifelogs can be beneficial to improve the quality of our life, as they can serve as tools for memory augmentation or for providing support to people with memory issues. In visual lifelogs, data are captured by cameras in the form of images or videos. However, a considerable amount of these images or videos are affected by different types of distortions or noise due to the non-controlled acquisition process. This article addresses the use of Blind Image Quality Assessment algorithms as a pre-processing approach in the retrieval of lifelogging images. As the amount of lifelog images has increased over the last few years, it is fundamental to find solutions to filter images in a lifelog data collection. We evaluate the impact of a Blind Image Quality Assessment algorithm by performing different retrieval experiments through a lifelogging system named MEMORIA. The results are promising and show that our approach can reduce the amount of images to process and retrieve in a lifelog data collection without losing valuable information, and provide to the user the most valuable images. By excluding a considerable amount of images in the pre-processing stage of a lifelogging system, its performance can be increased by saving time and resources.

References

[1]
Ahmed Alateeq, Mark Roantree, and Cathal Gurrin. 2022. Voxento 3.0: A Prototype Voice-Controlled Interactive Search Engine for Lifelog. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 43--47. https: //doi.org/10.1145/3512729.3533009
[2]
Soumyadeb Chowdhury, Philip J McParlane, Md Sadek Ferdous, and Joemon Jose. 2015. " My Day in Review" Visually Summarising Noisy Lifelog Data. In Proceedings of the 5th ACM on International Conference on Multimedia Retrieval. 607--610.
[3]
Chenyou Fan, Zehua Zhang, and David J Crandall. 2018. Deepdiary: Lifelogging image captioning and summarization. Journal of Visual Communication and Image Representation 55 (2018), 40--55.
[4]
Deepti Ghadiyaram and Alan C Bovik. 2015. Massive online crowdsourced study of subjective and objective picture quality. IEEE Transactions on Image Processing 25, 1 (2015), 372--387.
[5]
Cathal Gurrin, Björn Jónsson, Klaus Schöffmann, Duc-Tien Dang-Nguyen, Jakub Lokoč, Minh-Triet Tran, Wolfgang Hürst, Luca Rossetto, and Graham Healy. 2021. Introduction to the Fourth Annual Lifelog Search Challenge, LSC'21. In Proceedings of the 2021 International Conference on Multimedia Retrieval (Taipei, Taiwan) (ICMR '21). Association for Computing Machinery, New York, NY, USA, 690--691. https://doi.org/10.1145/3460426.3470945
[6]
Cathal Gurrin, Alan F Smeaton, Daragh Byrne, Neil O'Hare, Gareth JF Jones, and Noel O'Connor. 2008. An examination of a large visual lifelog. In Asia Information Retrieval Symposium. Springer, 537--542.
[7]
Cathal Gurrin, Alan F Smeaton, and Aiden R Doherty. 2014. Lifelogging: Personal big data. Foundations and trends in information retrieval 8, 1 (2014), 1--125.
[8]
Cathal Gurrin, Liting Zhou, Graham Healy, Björn Jónsson, Duc-Tien DangNguyen, Jakub Lokoč, Minh-Triet Tran, Wolfgang Hürst, Luca Rossetto, and Klaus Schöffmann. 2022. Introduction to the Fifth Annual Lifelog Search Challenge, LSC'22. In Proceedings of the 2022 International Conference on Multimedia Retrieval (Newark, NJ, USA) (ICMR '22). Association for Computing Machinery, New York, NY, USA, 685--687. https://doi.org/10.1145/3512527.3531439
[9]
Nhat Hoang-Xuan, Hoang-Phuc Trang-Trung, E-Ro Nguyen, Thanh-Cong Le, Mai-Khiem Tran, Tu-Khiem Le, Van-Tu Ninh, Cathal Gurrin, and Minh-Triet Tran. 2022. Flexible Interactive Retrieval SysTem 3.0 for Visual Lifelog Exploration at LSC 2022. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 20--26. https://doi.org/10.1145/3512729.3533013
[10]
Vlad Hosu, Hanhe Lin, Tamas Sziranyi, and Dietmar Saupe. 2020. KonIQ-10k: An Ecologically Valid Database for Deep Learning of Blind Image Quality Assessment. IEEE Transactions on Image Processing 29 (2020), 4041--4056. https://doi.org/10. 1109/TIP.2020.2967829
[11]
Feiyan Hu and Alan F Smeaton. 2018. Image aesthetics and content in selecting memorable keyframes from lifelogs. In International Conference on Multimedia Modeling. Springer, 608--619.
[12]
Amornched Jinda-Apiraksa, Jana Machajdik, and Robert Sablatnig. 2013. A Keyframe Selection of Lifelog Image Sequences. In MVA. Citeseer, 33--36.
[13]
Glenn Jocher, Ayush Chaurasia, Alex Stoken, Jirka Borovec, NanoCode012, Yonghye Kwon, TaoXie, Jiacong Fang, imyhxy, Kalen Michael, Lorna, Abhiram V, Diego Montes, Jebastin Nadar, Laughing, tkianai, yxNONG, Piotr Skalski, Zhiqiang Wang, Adam Hogan, Cristi Fati, Lorenzo Mammana, AlexWang1900, Deep Patel, Ding Yiwei, Felix You, Jan Hajek, Laurentiu Diaconu, and Mai Thanh Minh. 2022. ultralytics/yolov5: v6.1 - TensorRT, TensorFlow Edge TPU and OpenVINO Export and Inference. https://doi.org/10.5281/zenodo.6222936
[14]
Marco Leonardi, Paolo Napoletano, Raimondo Schettini, and Alessandro Rozza. 2021. No Reference, Opinion Unaware Image Quality Assessment by Anomaly Detection. Sensors 21, 3 (2021), 994.
[15]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. 2014. Microsoft coco: Common objects in context. In European conference on computer vision. Springer, 740--755.
[16]
Photchara Ratsamee, Yasushi Mae, Amornched Jinda-Apiraksa, Mitsuhiro Horade, Kazuto Kamiyama, Masaru Kojima, and Tatsuo Arai. 2015. Keyframe selection framework based on visual and excitement features for lifelog image sequences. International Journal of Social Robotics 7, 5 (2015), 859--874.
[17]
Ricardo Ribeiro, Alina Trifan, and António J R Neves. 2022. Lifelog Retrieval From Daily Digital Data: Narrative Review. JMIR Mhealth Uhealth 10, 5 (2 May 2022), e30517. https://doi.org/10.2196/30517
[18]
Ricardo F Ribeiro, António JR Neves, and José Luis Oliveira. 2020. Image selection based on low level properties for lifelog moment retrieval. In Twelfth International Conference on Machine Vision (ICMV 2019), Vol. 11433. International Society for Optics and Photonics, 1143303.
[19]
Ricardo Ribiero, Alina Trifan, and Antonio J. R. Neves. 2022. MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2022. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 8--13. https://doi.org/10.1145/3512729.3533011
[20]
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV) 115, 3 (2015), 211--252. https: //doi.org/10.1007/s11263-015-0816-y
[21]
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A Alemi. 2017. Inception-v4, inception-resnet and the impact of residual connections on learning. In Thirty-first AAAI conference on artificial intelligence.
[22]
Ly-Duyen Tran, Manh-Duy Nguyen, Binh Nguyen, Hyowon Lee, Liting Zhou, and Cathal Gurrin. 2022. E-Myscéal: Embedding-Based Interactive Lifelog Retrieval System for LSC'22. In Proceedings of the 5th Annual on Lifelog Search Challenge (Newark, NJ, USA) (LSC '22). Association for Computing Machinery, New York, NY, USA, 32--37. https://doi.org/10.1145/3512729.3533012
[23]
Shaoping Xu, Shunliang Jiang, and Weidong Min. 2017. No-reference/blind image quality assessment: a survey. IETE Technical Review 34, 3 (2017), 223--245
[24]
I. Zeki Yalniz, Hervé Jégou, Kan Chen, Manohar Paluri, and Dhruv Mahajan. 2019. Billion-scale semi-supervised learning for image classification. arXiv:1905.00546 [cs.CV]
[25]
Bolei Zhou, Agata Lapedriza, Aditya Khosla, Aude Oliva, and Antonio Torralba. 2017. Places: A 10 million image database for scene recognition. IEEE transactions on pattern analysis and machine intelligence 40, 6 (2017), 1452--1464

Cited By

View all
  • (2023)MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023Proceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593099(18-23)Online publication date: 12-Jun-2023

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences
IMuR '22: Proceedings of the 2nd International Workshop on Interactive Multimedia Retrieval
October 2022
54 pages
ISBN:9781450394970
DOI:10.1145/3552467
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. blind image quality assessment
  2. deep learning
  3. image annotation
  4. image processing
  5. lifelog
  6. lifelogging

Qualifiers

  • Research-article

Conference

MM '22
Sponsor:

Acceptance Rates

IMuR '22 Paper Acceptance Rate 6 of 6 submissions, 100%;
Overall Acceptance Rate 6 of 6 submissions, 100%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)78
  • Downloads (Last 6 weeks)6
Reflects downloads up to 27 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2023)MEMORIA: A Memory Enhancement and MOment RetrIeval Application for LSC 2023Proceedings of the 6th Annual ACM Lifelog Search Challenge10.1145/3592573.3593099(18-23)Online publication date: 12-Jun-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media