A Mid-air Multi-touch Interface using an Ultrafast Pan-tilt Camera
Authors: 
Yuta Abe, Taishi Iriyama, 
Takashi Komuro, Kohei Shimasaki, Idaku IshiiAuthors Info & Claims
Article No.: 19, Pages 1 - 6
Abstract
We propose an interface that enables bare hand interaction with fingers of both hands even from a distance, using an ultrafast pan-tilt camera and a wide view camera. In this interface, a user can perform multi-touch interaction with a large display using mid-air finger motions. In order to acquire accurate finger motions required for multi-touch interaction, a wide view camera is used to detect the positions of the user’s hands, and an ultrafast pan-tilt camera is utilized to obtain the images around the hands with high spatial resolution. In the experiments, we evaluated the accuracy of acquired finger motions, and showed that the proposed interface can capture both large motions of the user’s whole body and small motions of the fingers with high accuracy. We also implemented two example applications, a drawing application and a photo layout application, using the proposed interface.
Supplemental Material
- Download
- 16.77 MB
- Transcript
- Download
- 34.14 MB
- Transcript
- Download
- 115.16 MB
- Transcript
References
[1]
Tadayoshi Aoyama, Liang Li, Mingjun Jiang, Kazuki Inoue, Takeshi Takaki, Idaku Ishii, Hua Yang, Chikako Umemoto, Hiroshi Matsuda, Makoto Chikaraishi, 2017. Vibration Sensing of a Bridge Model using a Multithread Active Vision System. IEEE/ASME Transactions on Mechatronics 23, 1 (2017), 179–189.
[2]
Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. 2021. OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields. IEEE Transactions on Pattern Analysis and Machine Intelligence 43, 1 (2021), 172–186.
[3]
Youness Choubik and Abdelhak Mahmoudi. 2016. Machine learning for real time poses classification using kinect skeleton data. In Proceedings of 2016 13th International Conference on Computer Graphics, Imaging and Visualization (CGiV). IEEE, 307–311.
[4]
Naina Dhingra, Eugenio Valli, and Andreas Kunz. 2020. Recognition and localisation of pointing gestures using a RGB-D camera. In Proceedings of International Conference on Human-Computer Interaction. Springer, 205–212.
[5]
Shaopeng Hu, Kohei Shimasaki, Mingjun Jiang, Taku Senoo, and Idaku Ishii. 2021. A Simultaneous Multi-Object Zooming System Using an Ultrafast Pan-Tilt Camera. IEEE Sensors Journal 21, 7 (2021), 9436–9448.
[6]
Chang Liu and Tamás Szirányi. 2021. Real-time human detection and gesture recognition for on-board uav rescue. Sensors 21, 6 (2021), 2180.
[7]
JQ Liu, Ryoma Fujii, Tomoko Tateyama, Yutaro Iwamoto, and YW Chen. 2017. Kinect-based gesture recognition for touchless visualization of medical images. International Journal of Computer and Electrical Engineering 9, 2 (2017), 421–429.
[8]
Alessandro Manzi, Filippo Cavallo, and Paolo Dario. 2017. A neural network approach to human posture classification and fall detection using rgb-d camera. In Proceedings of Ambient Assisted Living: Italian Forum 2016 7. 127–139.
[9]
Kotaro Maruyama, Jungpil Shin, Cheol Min Kim, and Chin-Ling Chen. 2017. User authentication using leap motion. In Proceedings of the International Conference on Research in Adaptive and Convergent Systems. 213–216.
[10]
Ville Mäkelä, Jobin James, Tuuli Keskinen, Jaakko Hakulinen, and Markku Turunen. 2017. “It’s Natural to Grab and Pull": Retrieving Content from Large Displays Using Mid-Air Gestures. IEEE Pervasive Computing 16, 3 (2017), 70–77.
[11]
Joao C Neves, Juan C Moreno, Silvio Barra, and Hugo Proença. 2015. Acquiring High-Resolution Face Images in Outdoor Environments: A Master-Slave Calibration Algorithm. In In Proceedings of 2015 IEEE 7th International Conference on Biometrics Theory, Applications and Systems (BTAS). 1–8.
[12]
Cristina Nuzzi, Stefano Ghidini, Roberto Pagani, Simone Pasinetti, Gabriele Coffetti, and Giovanna Sansoni. 2020. Hands-Free: a robot augmented reality teleoperation system. In Proceedings of 2020 17th International Conference on Ubiquitous Robots (UR). 617–624.
[13]
Kohei Okumura, Hiromasa Oku, and Masatoshi Ishikawa. 2011. High-speed Gaze Controller for Millisecond-Order Pan/Tilt Camera. In Proceedings of 2011 IEEE International Conference on Robotics and Automation. 6186–6191.
[14]
Georgios Th Papadopoulos, Apostolos Axenopoulos, and Petros Daras. 2014. Real-time skeleton-tracking-based human action recognition using kinect data. In Proceedings of MultiMedia Modeling: 20th Anniversary International Conference, MMM 2014, Dublin, Ireland, January 6-10, 2014, Proceedings, Part I 20. Springer, 473–483.
[15]
Junchen Shen, Yanlin Luo, Zhongke Wu, Yun Tian, and Qingqiong Deng. 2016. CUDA-based real-time hand gesture interaction and visualization for CT volume dataset using leap motion. The Visual Computer 32, 3 (2016), 359–370.
[16]
Michal Tölgyessy, Martin Dekan, František Duchoň, Jozef Rodina, Peter Hubinskỳ, and L’uboš Chovanec. 2017. Foundations of visual linear human–robot interaction via pointing gesture navigation. International Journal of Social Robotics 9 (2017), 509–523.
[17]
Yu Xiao and Qingjin Peng. 2017. A hand gesture-based interface for design review using leap motion controller. In DS 87-8 Proceedings of the 21st International Conference on Engineering Design (ICED 17) Vol 8: Human Behaviour in Design, Vancouver, Canada, 21-25.08. 2017. 239–248.
[18]
Yiliang Xu and Dezhen Song. 2010. Systems and algorithms for autonomous and scalable crowd surveillance using robotic PTZ cameras assisted by a wide-angle camera. Autonomous Robots 29, 1 (2010), 53–66.
Index Terms
- A Mid-air Multi-touch Interface using an Ultrafast Pan-tilt Camera
Recommendations
Multi-scenario gesture recognition using Kinect
CGAMES '12: Proceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)Hand gesture recognition (HGR) is an important research topic because some situations require silent communication with sign languages. Computational HGR systems assist silent communication, and help people learn a sign language. In this article, a ...
Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human InteractionThe natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...
Image-based pan-tilt camera control in a multi-camera surveillance environment
ICME '03: Proceedings of the 2003 International Conference on Multimedia and Expo - Volume 2In automated surveillance systems with multiple cameras, the system must be able to position the cameras accurately. Each camera must be able to pan-tilt such that an object detected in the scene is in a vantage position in the camera's image plane and ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Work in progress
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI 2025
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 299Total Downloads
- Downloads (Last 12 months)299
- Downloads (Last 6 weeks)85
Reflects downloads up to 03 Jan 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format