IRIS: Wireless ring for vision-based smart home interaction
Authors: 
Maruchi Kim, Antonio Glenn, 
Bandhav Veluri, Yunseo Lee, Eyoel Gebre, Aditya Bagaria, 
Shwetak Patel, 
Shyamnath GollakotaAuthors Info & Claims
Article No.: 114, Pages 1 - 16
Abstract
Integrating cameras into wireless smart rings has been challenging due to size and power constraints. We introduce IRIS, the first wireless vision-enabled smart ring system for smart home interactions. Equipped with a camera, Bluetooth radio, inertial measurement unit (IMU), and an onboard battery, IRIS meets the small size, weight, and power (SWaP) requirements for ring devices. IRIS is context-aware, adapting its gesture set to the detected device, and can last for 16-24 hours on a single charge. IRIS leverages the scene semantics to achieve instance-level device recognition. In a study involving 23 participants, IRIS consistently outpaced voice commands, with a higher proportion of participants expressing a preference for IRIS over voice commands regarding toggling a device’s state, granular control, and social acceptability. Our work pushes the boundary of what is possible with ring form-factor devices, addressing system challenges and opening up novel interaction capabilities.
Supplemental Material
MP4 File
Presentation and Demo Video
- Download
- 243.93 MB
MP4 File
User Study Participant Video
- Download
- 54.22 MB
References
[1]
Amr Alanwar, Moustafa Alzantot, Bo-Jhang Ho, Paul Martin, and Mani Srivastava. 2016. SeleCon: Scalable IoT Device Selection and Control Using Hand Gestures. In Proceedings of the 10th ACM Conference on Embedded Systems for Energy-Efficient Buildings, Vol. 2017. IoTDI 2017 (2017), Log Angeles, CA, USA, 107–114. https://doi.org/10.1145/3054977.3054981 29683151
[2]
Amazon. 2024. Amazon Alexa Voice AI | Alexa Developer Official Site. Amazon Alexa. https://developer.amazon.com/en-US/alexa
[3]
Apple. 2024. HomePod. https://www.apple.com/homepod/.
[4]
Roger Boldu, Alexandru Dancu, Denys J.C. Matthies, Thisum Buddhika, Shamane Siriwardhana, and Suranga Nanayakkara. 2018. FingerReader2.0: Designing and Evaluating a Wearable Finger-Worn Camera to Assist People with Visual Impairments while Shopping. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 3, Article 94 (sep 2018), 19 pages. https://doi.org/10.1145/3264904
[5]
Liwei Chan, Yi-Ling Chen, Chi-Hao Hsieh, Rong-Hao Liang, and Bing-Yu Chen. 2015. CyclopsRing: Enabling Whole-Hand and Context-Aware Interactions Through a Fisheye Ring. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology (Charlotte, NC, USA) (UIST ’15). Association for Computing Machinery, New York, NY, USA, 549–556. https://doi.org/10.1145/2807442.2807450
[6]
Kaifei Chen, Jonathan Fürst, John Kolb, Hyung-Sin Kim, Xin Jin, David E. Culler, and Randy H. Katz. 2018. SnapLink: Fast and Accurate Vision-Based Appliance Control in Large Commercial Buildings. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 129 (jan 2018), 27 pages. https://doi.org/10.1145/3161173
[7]
Rajkumar Darbar, Mainak Choudhury, and Vikalp Mullick. 2019. RingIoT: A Smart Ring Controlling Things in Physical Spaces., 2–9 pages.
[8]
Adrian A. de Freitas, Michael Nebeling, Xiang ’Anthony’ Chen, Junrui Yang, Akshaye Shreenithi Kirupa Karthikeyan Ranithangam, and Anind K. Dey. 2016. Snap-To-It: A User-Inspired Platform for Opportunistic Device Interactions. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5909–5920. https://doi.org/10.1145/2858036.2858177
[9]
Retail Dive. 2024. 27% Increase in Smart Home Adoption Since 2020: YouGov Report. https://www2.deloitte.com/us/en/insights/industry/telecommunications/connectivity-mobile-trends-survey/2023/smart-home-industry-adoption-trend.html
[10]
Daily Dot. 2022. Ring Zero: The Smart Logbar That Could Change How We Interact With Tech. https://www.dailydot.com/debug/ring-zero-smart-logbar-sxsw/
[11]
Yasuhiro Endo, Zheng Wang, J Bradley Chen, and Margo I Seltzer. 1996. Using latency to evaluate interactive system performance. ACM SIGOPS Operating Systems Review 30, si (1996), 185–199.
[12]
M. Fiala. 2005. ARTag, a fiducial marker system using digital techniques. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), Vol. 2. IEEE, San Diego, CA, USA, 590–596 vol. 2. https://doi.org/10.1109/CVPR.2005.74
[13]
Bogdan-Florin Gheran, Jean Vanderdonckt, and Radu-Daniel Vatavu. 2018. Gestures for Smart Rings: Empirical Results, Insights, and Design Implications. In Proceedings of the 2018 Designing Interactive Systems Conference (Hong Kong, China) (DIS ’18). Association for Computing Machinery, New York, NY, USA, 623–635. https://doi.org/10.1145/3196709.3196741
[14]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Redmond, WA, USA, 770–778.
[15]
Vikram Iyer, Ali Najafi, Johannes James, Sawyer Fuller, and Shyamnath Gollakota. 2020. Wireless steerable vision for live insects and insect-scale robots. Science robotics 5, 44 (2020), eabb0839. https://doi.org/10.1126/scirobotics.abb0839
[16]
Shilpi Jain, Sriparna Basu, Arghya Ray, and Ronnie Das. 2023. Impact of irritation and negative emotions on the performance of voice assistants: Netting dissatisfied customers’ perspectives. International Journal of Information Management 72 (2023), 102662. https://doi.org/10.1016/j.ijinfomgt.2023.102662
[17]
Shu-Jun Ji, Qing-Hua Ling, and Fei Han. 2023. An Improved Algorithm for Small Object Detection Based on YOLO v4 and Multi-scale Contextual Information. Computers and Electrical Engineering 105 (2023), 108490. https://doi.org/10.1016/j.compeleceng.2022.108490
[18]
Lei Jing, Zixue Cheng, Yinghui Zhou, Junbo Wang, and Tongjun Huang. 2013. Magic Ring: a self-contained gesture input device on finger. In Proceedings of the 12th International Conference on Mobile and Ubiquitous Multimedia (Luleå, Sweden) (MUM ’13). Association for Computing Machinery, New York, NY, USA, Article 39, 4 pages. https://doi.org/10.1145/2541831.2541875
[19]
Runchang Kang, Anhong Guo, Gierad Laput, Yang Li, and Xiang ’Anthony’ Chen. 2019. Minuet: Multimodal Interaction with an Internet of Things. In Symposium on Spatial User Interaction (New Orleans, LA, USA) (SUI ’19). Association for Computing Machinery, New York, NY, USA, Article 2, 10 pages. https://doi.org/10.1145/3357251.3357581
[20]
Xiaoyu Li, Shuqin Zeng, Yanwei Zhang, Ping Wan, and Jun Wang. 2012. Analysis and processing of pixel binning for color image sensor. EURASIP Journal on Advances in Signal Processing 2012, 1 (2012), 81. https://doi.org/10.1186/1687-6180-2012-81
[21]
Brian D. Mayton, Nan Zhao, Matt Aldrich, Nicholas Gillian, and Joseph A. Paradiso. 2013. WristQue: A personal sensor wristband. In 2013 IEEE International Conference on Body Sensor Networks (Cambridge, MA, USA). IEEE, Cambridge, MA, USA, 1–6. https://doi.org/10.1109/BSN.2013.6575483
[22]
Jay McGregor. 2024. Samsung Galaxy Ring: Release Date, Price, Design, Features. Forbes. https://www.forbes.com/sites/jaymcgregor/2024/03/11/samsung-galaxy-ring-release-date-price-design-features/?sh=308cc8f513bf
[23]
Kento Miyaoku, Anthony Tang, and Sidney Fels. 2007. C-Band: A Flexible Ring Tag System for Camera-Based User Interface. In Virtual Reality, Randall Shumaker (Ed.). Springer Berlin Heidelberg, Berlin, Heidelberg, 320–328.
[24]
Suranga Nanayakkara, Roy Shilkrot, Kian Peen Yeo, and Pattie Maes. 2013. EyeRing: a finger-worn input device for seamless interactions with our surroundings. In Proceedings of the 4th Augmented Human International Conference (Stuttgart, Germany) (AH ’13). Association for Computing Machinery, New York, NY, USA, 13–20. https://doi.org/10.1145/2459236.2459240
[25]
Jared Newman. 2022. The Smart Home Is Flailing as a Concept Because It Sucks. https://www.fastcompany.com/90660570/the-smart-home-is-flailing-as-a-concept-because-it-sucks
[26]
Jakob Nielsen. 1993. Smart Home Statistics. https://www.nngroup.com/articles/response-times-3-important-limits/.
[27]
Oberlo. 2024. Smart Home Statistics. https://www.oberlo.com/statistics/smart-home-market.
[28]
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. 2024. DINOv2: Learning Robust Visual Features without Supervision. arxiv:2304.07193 [cs.CV]
[29]
Oura. 2024. Oura Ring. https://ouraring.com/. Accessed: March 31, 2024.
[30]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. arxiv:2103.00020 [cs.CV]
[31]
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. 2016. You Only Look Once: Unified, Real-Time Object Detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, Seattle, WA, USA, 779–788. https://doi.org/10.1109/CVPR.2016.91
[32]
Leon Reicherts, Yvonne Rogers, Licia Capra, Ethan Wood, Tu Dinh Duong, and Neil Sebire. 2022. It’s Good to Talk: A Comparison of Using Voice Versus Screen-Based Interactions for Agent-Assisted Tasks. ACM Trans. Comput.-Hum. Interact. 29, 3, Article 25 (jan 2022), 41 pages. https://doi.org/10.1145/3484221
[33]
Ana Rodrigues, Rita Santos, Jorge Abreu, Pedro Beça, Pedro Almeida, and Sílvia Fernandes. 2019. Analyzing the performance of ASR systems: The effects of noise, distance to the device, age and gender. In Proceedings of the XX International Conference on Human Computer Interaction (Donostia, Gipuzkoa, Spain) (Interacción ’19). Association for Computing Machinery, New York, NY, USA, Article 8, 8 pages. https://doi.org/10.1145/3335595.3335635
[34]
Mia Sapienza. 2022. Are You Still Relying on Your Phone to Control Your Home?https://www.brilliant.tech/blogs/news/are-you-still-relying-on-your-phone-to-control-your-home
[35]
Nordic Semiconductor. 2022. Things You Should Know About Bluetooth Range. Nordic Semiconductor. https://blog.nordicsemi.com/getconnected/things-you-should-know-about-bluetooth-range
[36]
Roy Shilkrot, Jochen Huber, Wong Meng Ee, Pattie Maes, and Suranga Chandima Nanayakkara. 2015. FingerReader: A Wearable Device to Explore Printed Text on the Go. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 2363–2372. https://doi.org/10.1145/2702123.2702421
[37]
Lee Stearns, Uran Oh, Leah Findlater, and Jon E. Froehlich. 2018. TouchCam: Realtime Recognition of Location-Specific On-Body Gestures to Support Users with Visual Impairments. Proc. ACM Interact. Mob. Wearable Ubiquitous Technol. 1, 4, Article 164 (jan 2018), 23 pages. https://doi.org/10.1145/3161416
[38]
George Stetten, Roberta Klatzky, Brock Nichol, John Galeotti, Kenneth Rockot, Kimberly Zawrotny, David Weiser, Nathan Sendgikoski, and Samantha Horvath. 2007. Fingersight: Fingertip Visual Haptic Sensing and Control. In 2007 IEEE International Workshop on Haptic, Audio and Visual Environments and Games. IEEE, Ottawa, ON, Canada, 80–83. https://doi.org/10.1109/HAVE.2007.4371592
[39]
Google Store. 2024. How to Set Up a Smart Home. Google. Accessed: March 31, 2024.
[40]
Mingxing Tan and Quoc Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, Mountain View, CA, USA, 6105–6114. https://proceedings.mlr.press/v97/tan19a.html
[41]
Punch Through. 2022. Maximizing BLE Throughput on iOS and Android. PunchThrough. https://punchthrough.com/maximizing-ble-throughput-on-ios-and-android/#: :text=It%20is%20important%20to%20know,per%20connection%20event%20in%20Android
[42]
Ultrahuman. 2024. Ultrahuman. https://www.ultrahuman.com//. Accessed: March 31, 2024.
[43]
Ultralytics. 2024. YOLOv8. https://github.com/ultralytics/yolov8. Accessed: 2024-03-31.
[44]
Radu-Daniel Vatavu and Laura-Bianca Bilius. 2021. GestuRING: A Web-based Tool for Designing Gesture Input with Rings, Ring-Like, and Ring-Ready Devices. In The 34th Annual ACM Symposium on User Interface Software and Technology (Virtual Event, USA) (UIST ’21). Association for Computing Machinery, New York, NY, USA, 710–723. https://doi.org/10.1145/3472749.3474780
[45]
Bandhav Veluri, Collin Pernu, Ali Saffari, Joshua Smith, Michael Taylor, and Shyamnath Gollakota. 2023. NeuriCam: Key-Frame Video Super-Resolution and Colorization for IoT Cameras. Association for Computing Machinery, New York, NY, USA, Chapter 25, 1–17. https://doi.org/10.1145/3570361.3592523
[46]
P. K.A. Wollner, P. M. Langdon, T. Goldhaber, I. M. Hosking, A. Mieczakowski, and P. J. Clarkson. 2012. Evaluation of setup procedures on mobile devices based on users’ initial experience. In NordDesign 2012 - Proceedings of the 9th NordDesign Conference, Poul Kyvsgaard Hansen, John Rasmussen, Kaj A. Jorgensen, and Christian Tollestrup (Eds.). Center for Industrial Production, Aalborg University and Design Society, University of Strathclyde, Aalborg, Denmark, 1–8. 9th NordDesign Conference, NordDesign 2012 ; Conference date: 22-08-2012 Through 24-08-2012.
[47]
Yoonjong Yoo, Jaehyun Im, and Joonki Paik. 2015. Low-Light Image Enhancement Using Adaptive Digital Pixel Binning. Sensors 15, 7 (2015), 14917–14931. https://doi.org/10.3390/s150714917
[48]
Sang Ho Yoon, Yunbo Zhang, Ke Huo, and Karthik Ramani. 2016. TRing: Instant and Customizable Interactions with Objects Using an Embedded Magnet and a Finger-Worn Device. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (Tokyo, Japan) (UIST ’16). Association for Computing Machinery, New York, NY, USA, 169–181. https://doi.org/10.1145/2984511.2984529
[49]
Tengxiang Zhang, Xin Zeng, Yinshuai Zhang, Ke Sun, Yuntao Wang, and Yiqiang Chen. 2020. ThermalRing: Gesture and Tag Inputs Enabled by a Thermal Imaging Smart Ring. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376323
Index Terms
- IRIS: Wireless ring for vision-based smart home interaction
Recommendations
Smart home context awareness based on Smart and Innovative Cities
EANN '15: Proceedings of the 16th International Conference on Engineering Applications of Neural Networks (INNS)In the emerging Smart Cities - Smart Homes computing paradigms developing a formalization for context information is increasingly important. In the present paper, basedon the EU FIRE research project "Social and Smart" we aim to formalize and build a ...
Smart Home Agents and Devices of Today and Tomorrow: Surveying Use and Desires
HAI '21: Proceedings of the 9th International Conference on Human-Agent InteractionHow are people using current smart home technologies, and how do they conceptualize future ones that are more interconnected and more capable than those available today? We deployed an online survey study to 150 participants to investigate use of and ...
Enablement of IoT Based Context-Aware Smart Home with Fog Computing
The recent advent of Internet of Things IoT, has given rise to a plethora of smart verticals-smart homes being one of them. Smart Home is a classic example of IoT, wherein smart appliances connected via home gateways constitute a local home network to ...
Comments
Please enable JavaScript to view thecomments powered by Disqus.Information & Contributors
Information
Published In
October 2024
2334 pages
Copyright © 2024 Owner/Author.
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike International 4.0 License.
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 October 2024
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Funding Sources
Conference
UIST '24
UIST '24: The 37th Annual ACM Symposium on User Interface Software and Technology
October 13 - 16, 2024
PA, Pittsburgh, USA
Acceptance Rates
Overall Acceptance Rate 561 of 2,567 submissions, 22%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 585Total Downloads
- Downloads (Last 12 months)585
- Downloads (Last 6 weeks)187
Reflects downloads up to 27 Jan 2025
Other Metrics
Citations
View Options
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML FormatLogin options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in