Abstract
Visual lifelogging using wearable cameras accumulates large amounts of image data. To make them useful they are typically structured into events corresponding to episodes which occur during the wearer’s day. These events can be represented as a visual storyboard, a collection of chronologically ordered images which summarise the day’s happenings. In previous work, little attention has been paid to how to select the representative keyframes for a lifelogged event, apart from the fact that the image should be of good quality in terms of absence of blurring, motion artifacts, etc. In this paper we look at image aesthetics as a characteristic of wearable camera images. We show how this can be used in combination with content analysis and temporal offsets, to offer new ways for automatically selecting wearable camera keyframes. In this paper we implement several variations of the keyframe selection method and illustrate how it works using a publicly-available lifelog dataset.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
Dhamija, R., Perrig, A.: Deja-Vu a user study: using images for authentication. In: USENIX Security Symposium, vol. 9, p. 4 (2000)
Dhar, S., Ordonez, V., Berg, T.L.: High level describable attributes for predicting aesthetics and interestingness. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1657–1664. IEEE (2011)
Doherty, A.R., Moulin, C.J.A., Smeaton, A.F.: Automatically assisting human memory: a SenseCam browser. Memory 19(7), 785–795 (2011)
Doherty, A.R., Smeaton, A.F.: Automatically segmenting lifelog data into events. In: 2008 9th International Workshop on Image Analysis for Multimedia Interactive Services, pp. 20–23, May 2008
Gurrin, C., Joho, H., Hopfgartner, F., Zhou, L., Albatal, R.: NTCIR lifelog: the first test collection for lifelog research. In: Proceedings of 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2016, pp. 705–708. ACM, New York (2016)
Gurrin, C., Smeaton, A.F., Byrne, D., O’Hare, N., Jones, G.J.F., O’Connor, N.: An examination of a large visual lifelog. In: Li, H., Liu, T., Ma, W.-Y., Sakai, T., Wong, K.-F., Zhou, G. (eds.) AIRS 2008. LNCS, vol. 4993, pp. 537–542. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-68636-1_60
Gurrin, C., Smeaton, A.F., Doherty, A.R.: Lifelogging: personal big data. Found. Trends Inf. Retr. 8(1), 1–125 (2014)
Harvey, M., Langheinrich, M., Ward, G.: Remembering through lifelogging: a survey of human memory augmentation. Pervasive Mob. Comput. 27, 14–26 (2016)
Isola, P., Parikh, D., Torralba, A., Oliva, A., Understanding the intrinsic memorability of images. In: Shawe-Taylor, J., Zemel, R.S., Bartlett, P.L., Pereira, F., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems, vol. 24, pp. 2429–2437. Curran Associates Inc. (2011)
Isola, P., Xiao, J., Parikh, D., Torralba, A., Oliva, A.: What makes a photograph memorable? IEEE Trans. Pattern Anal. Mach. Intell. 36(7), 1469–1482 (2014)
Kätsyri, J., Ravaja, N., Salminen, M.: Aesthetic images modulate emotional responses to reading news messages on a small screen: a psychophysiological investigation. Int. J. Hum. Comput. Stud. 70(1), 72–87 (2012)
Khosla, A., Xiao, J., Isola, P., Torralba, A., Oliva, A.: Image memorability and visual inception. In: SIGGRAPH Asia 2012 Technical Briefs, SA 2012, pp. 35:1–35:4. ACM, New York (2012)
Lu, X., Lin, Z., Jin, H., Yang, J., Wang, J.Z.: Rating image aesthetics using deep learning. IEEE Trans. Multimedia 17(11), 2021–2034 (2015)
Mai, L., Jin, H., Liu, F.: Composition-preserving deep photo aesthetics assessment. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 497–506 (2016)
Pan, J., Sayrol, E., Giro-i Nieto, X., McGuinness, K., O’Connor, N.E.: Shallow and deep convolutional networks for saliency prediction. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 598–606 (2016)
Piasek, P., Irving, K., Smeaton, A.F.: SenseCam intervention based on cognitive stimulation therapy framework for early-stage dementia. In: 2011 5th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth) and Workshops, pp. 522–525, May 2011
Rodden, K., Wood, K.R.: How do people manage their digital photographs? In: Proceedings of SIGCHI Conference on Human Factors in Computing Systems, CHI 2003, pp. 409–416. ACM, New York (2003)
Silva, A.R., Pinho, M.S., Macedo, L., Moulin, C.J.A.: A critical review of the effects of wearable cameras on memory. Neuropsychol. Rehabil. 26(1), 1–25 (2016). PMID: 26732623
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this paper
Cite this paper
Hu, F., Smeaton, A.F. (2018). Image Aesthetics and Content in Selecting Memorable Keyframes from Lifelogs. In: Schoeffmann, K., et al. MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science(), vol 10704. Springer, Cham. https://doi.org/10.1007/978-3-319-73603-7_49
Download citation
DOI: https://doi.org/10.1007/978-3-319-73603-7_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-73602-0
Online ISBN: 978-3-319-73603-7
eBook Packages: Computer ScienceComputer Science (R0)