More Web Proxy on the site http://driver.im/

research-article

Public Access

CoSummary: adaptive fast-forwarding for surgical videos by detecting collaborative scenes using hand regions and gaze positions

Authors:

Irshad Abibouraguimane,

Kakeru Hagihara,

Tetsu Hayashida,

Maki SugimotoAuthors Info & Claims

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Pages 580 - 590

https://doi.org/10.1145/3301275.3302284

Published: 17 March 2019 Publication History

Abstract

This paper presents CoSummary, an adaptive video fast-forwarding technique for browsing surgical videos recorded by wearable cameras. Current wearable technologies allow us to record complex surgical skills, however, an efficient browsing technique for these videos is not well established. In order to assist browsing surgical videos, our study focuses on adaptively changing playback speeds through the learning and detecting collaborative scenes based on surgeon hand placement and gaze information. Our evaluation shows that the proposed method is able to highlight important collaborative scenes and skip less important scenes during surgical procedures. We have also performed a subjective study with surgeons in order to have professional feedback. The results confirmed the effectiveness of the proposed method in comparison to uniform video fast-forwarding.

References

[1]

Kai-Yin Cheng, Sheng-Jie Luo, Bing-Yu Chen, and Hao-Hua Chu. 2009. Smart-Player: User-centric Video Fast-forwarding. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09). ACM, New York, NY, USA, 789--798.

Digital Library

[2]

Kurara Fukumoto, Tsutomu Terada, and Masahiko Tsukamoto. 2013. A Smile/Laughter Recognition Mechanism for Smile-based Life Logging. In Proceedings of the 4th Augmented Human International Conference (AH '13). ACM, New York, NY, USA, 213--220.

Digital Library

[3]

Susan R. Fussell, Leslie D. Setlock, and Elizabeth M. Parker. 2003. Where do helpers look?: gaze targets during collaborative physical tasks. In Extended abstracts of the 2003 Conference on Human Factors in Computing Systems (CHI'03). ACM, New York, NY, USA, 768--769.

Digital Library

[4]

Keita Higuchi, Ryo Yonetani, and Yoichi Sato. 2016. Can Eye Help You?: Effects of Visualizing Eye Fixations on Remote Collaboration Scenarios for Physical Tasks. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5180--5190.

Digital Library

[5]

Keita Higuchi, Ryo Yonetani, and Yoichi Sato. 2017. EgoScanning: Quickly Scanning First-Person Videos with Egocentric Elastic Timelines. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6536--6546.

Digital Library

[6]

Chien-Ming Huang, Sean Andrist, Allison Sauppe, and Bilge Mutlu. 2015. Using Gaze Patterns to Predict Task Intent in Collaboration. Frontiers in Psychology 6 (July 2015).

[7]

Peter Kasarskis, Jennifer Stehwien, Joey Hickox, Anthony Aretz, and Chris Wickens. 2001. Comparison of expert and novice scan behaviors during VFR flight. In Proceedings of the 11th International Symposium on Aviation Psychology, Vol. 6.

[8]

Adam Kendon. 1967. Some functions of gaze-direction in social interaction. Acta psychologica 26 (1967), 22--63.

[9]

Kris M. Kitani and Cheng Li. 2013. Pixel-level Hand Detection in Ego-Centric Videos. In Conference on Computer Vision and Pattern Recognition.

Digital Library

[10]

Harold L Kundel, Calvin F Nodine, Emily F Conant, and Susan P Weinstein. 2007. Holistic component of image perception in mammogram interpretation: gaze-tracking study. Radiology 242, 2 (2007), 396--402.

[11]

Wen-Nung Lie and Kuo-Chiang Hsu. 2008. Video Summarization Based on Semantic Feature Analysis and User Preference. 2008 IEEE International Conference on Sensor Networks, Ubiquitous, and Trustworthy Computing (2008), 486--491.

Digital Library

[12]

Isao Otsuka, Kazuhiko Nakane, Ajay Divakaran, Keiji Hatanaka, and Masaharu Ogawa. 2005. A highlight scene detection and video summarization system using audio feature for a personal video recorder. IEEE Transactions on Consumer Electronics 51, 1 (2005), 112--116.

Digital Library

[13]

Mona Wanda Schmidt, Mirco Friedrich, Karl-Friedrich Kowalewski, Javier De La Garza, Thomas Bruckner, Beat-Peter MÃijller-Stich, and Felix Nickel. 2017. Learning from the surgeon's real perspective - First-person view versus laparoscopic view in e-learning for training of surgical skills? Study protocol for a randomized controlled trial. International Journal of Surgery Protocols 3 (2017), 7 -- 13.

[14]

Tony Tien, Philip H Pucher, Mikael H Sodergren, Kumuthan Sriskandarajah, Guang-Zhong Yang, and Ara Darzi. 2014. Eye tracking for skills assessment and training: a systematic review. journal of surgical research 191, 1 (2014), 169--178.

[15]

Joan N Vickers. 1992. Gaze control in putting. Perception 21, 1 (1992), 117--132.

[16]

Stine Vogt and Svein Magnussen. 2007. Expertise in pictorial perception: eye-movement patterns and visual memory in artists and laymen. Perception 36, 1 (2007), 91--100.

[17]

Hossam M Zawbaa, Nashwa El-Bendary, Aboul Ella Hassanien, and Ajith Abraham. 2011a. SVM-based soccer video summarization system. In Third World Congress on Nature and Biologically Inspired Computing (NaBIC). IEEE, 7--11.

[18]

Hossam M Zawbaa, Nashwa El-Bendary, Aboul Ella Hassanien, and Tai-hoon Kim. 2011b. Machine learning-based soccer video summarization system. In Multimedia, Computer Graphics and Broadcasting. Springer, 19--28.

[19]

Yanxia Zhang, Ken Pfeuffer, Ming Ki Chong, Jason Alexander, Andreas Bulling, and Hans Gellersen. 2017. Look together: using gaze for assisting co-located collaborative search. Personal and Ubiquitous Computing 21, 1 (2017), 173--186.

Digital Library

Cited By

Tomita HIenaga NKajita HHayashida TSugimoto M(2024)An analysis on the effect of body tissues and surgical tools on workflow recognition in first person surgical videosInternational Journal of Computer Assisted Radiology and Surgery10.1007/s11548-024-03074-619:11(2195-2202)Online publication date: 27-Feb-2024
https://doi.org/10.1007/s11548-024-03074-6
Yagi TNishiyasu TKawasaki KMatsuki MSato Y(2022)GO-Finder: A Registration-free Wearable System for Assisting Users in Finding Lost Hand-held ObjectsACM Transactions on Interactive Intelligent Systems10.1145/351926812:4(1-29)Online publication date: 4-Nov-2022
https://dl.acm.org/doi/10.1145/3519268
Akgul TUgur DBegen A(2022)Automated Adaptive Playback for Encoder-Adjudicated Live Sports2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR54900.2022.00065(330-335)Online publication date: Aug-2022
https://doi.org/10.1109/MIPR54900.2022.00065
Show More Cited By

Index Terms

CoSummary: adaptive fast-forwarding for surgical videos by detecting collaborative scenes using hand regions and gaze positions
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Activity recognition and understanding
        Video summarization
2. Social and professional topics
  1. Computing / technology policy
    1. Medical information policy
      1. Medical records

Recommendations

Inferring target locations from gaze data: a smartphone study
ETRA '19: Proceedings of the 11th ACM Symposium on Eye Tracking Research & Applications

Although smartphones are widely used in everyday life, studies of viewing behavior mainly employ desktop computers. This study examines whether closely spaced target locations on a smartphone can be decoded from gaze. Subjects wore a head-mounted eye ...
HGR‐Net: a fusion network for hand gesture segmentation and recognition

We propose a two‐stage convolutional neural network (CNN) architecture for robust recognition of hand gestures, called HGR‐Net, where the first stage performs accurate semantic segmentation to determine hand regions, and the second stage identifies the ...
Low-cost head position tracking for gaze point estimation
PETRA '12: Proceedings of the 5th International Conference on PErvasive Technologies Related to Assistive Environments

In this paper, we present a low-cost solution for real-time tracking of a human user's head position with respect to a video display source for eye gaze estimation in an assistive setting. The solution utilizes a wearable headset equipped with sensors ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

March 2019

713 pages

ISBN:9781450362726

DOI:10.1145/3301275

General Chairs:
Wai-Tat Fu,
Shimei Pan,
Program Chairs:
Oliver Brdiczka,
Polo Chau,
Gaelle Calvary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Conference

IUI '19

Sponsor:

IUI '19: 24th International Conference on Intelligent User Interfaces

March 17 - 20, 2019

California, Marina del Ray

Acceptance Rates

IUI '19 Paper Acceptance Rate 71 of 282 submissions, 25%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

10
Total Citations
View Citations
512
Total Downloads

Downloads (Last 12 months)61
Downloads (Last 6 weeks)9

Reflects downloads up to 11 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Tomita HIenaga NKajita HHayashida TSugimoto M(2024)An analysis on the effect of body tissues and surgical tools on workflow recognition in first person surgical videosInternational Journal of Computer Assisted Radiology and Surgery10.1007/s11548-024-03074-619:11(2195-2202)Online publication date: 27-Feb-2024
https://doi.org/10.1007/s11548-024-03074-6
Yagi TNishiyasu TKawasaki KMatsuki MSato Y(2022)GO-Finder: A Registration-free Wearable System for Assisting Users in Finding Lost Hand-held ObjectsACM Transactions on Interactive Intelligent Systems10.1145/351926812:4(1-29)Online publication date: 4-Nov-2022
https://dl.acm.org/doi/10.1145/3519268
Akgul TUgur DBegen A(2022)Automated Adaptive Playback for Encoder-Adjudicated Live Sports2022 IEEE 5th International Conference on Multimedia Information Processing and Retrieval (MIPR)10.1109/MIPR54900.2022.00065(330-335)Online publication date: Aug-2022
https://doi.org/10.1109/MIPR54900.2022.00065
Kajita H(2021)Surgical Video Recording and Application of Deep Learning for Open SurgeryOpen surgeryにおける術野撮影と深層学習の利用Journal of Japan Society of Computer Aided Surgery10.5759/jscas.23.5923:2(59-64)Online publication date: 2021
https://doi.org/10.5759/jscas.23.59
Kaluarachchi TReis ANanayakkara S(2021)A Review of Recent Deep Learning Approaches in Human-Centered Machine LearningSensors10.3390/s2107251421:7(2514)Online publication date: 3-Apr-2021
https://doi.org/10.3390/s21072514
Aladag OUgur DAkcay MBegen AAlay ÖHsu CBegen A(2021)Content-Aware Playback Speed Control for Low-Latency Live Streaming of SportsProceedings of the 12th ACM Multimedia Systems Conference10.1145/3458305.3478437(344-349)Online publication date: 24-Jun-2021
https://dl.acm.org/doi/10.1145/3458305.3478437
Bafna TBækgaard PPaulin Hansen J(2021)EyeTell: Tablet-based Calibration-free Eye-typing using Smooth-pursuit movementsACM Symposium on Eye Tracking Research and Applications10.1145/3448018.3458015(1-6)Online publication date: 25-May-2021
https://dl.acm.org/doi/10.1145/3448018.3458015
Yoshida KHachiuma RTomita HPan JKitani KKajita HHayashida TSugimoto M(2021)Spatiotemporal Video Highlight by Neural Network Considering Gaze and Hands of Surgeon in Egocentric Surgical VideosJournal of Medical Robotics Research10.1142/S2424905X2141001407:01Online publication date: 22-Dec-2021
https://doi.org/10.1142/S2424905X21410014
Yoon BKim HOh SWoo W(2020)Evaluating Remote Virtual Hands Models on Social Presence in Hand-based 3D Remote Collaboration2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR50242.2020.00080(520-532)Online publication date: Nov-2020
https://doi.org/10.1109/ISMAR50242.2020.00080
Lee GAhn SHoff WBillinghurst M(2020)Enhancing First-Person View Task Instruction Videos with Augmented Reality Cues2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR50242.2020.00078(498-508)Online publication date: Nov-2020
https://doi.org/10.1109/ISMAR50242.2020.00078

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Table of Contents