[go: up one dir, main page]
More Web Proxy on the site http://driver.im/ skip to main content
research-article
Open access

Captivates: A Smart Eyeglass Platform for Across-Context Physiological Measurement

Published: 14 September 2021 Publication History

Abstract

We present Captivates, an open-source smartglasses system designed for long-term, in-the-wild psychophysiological monitoring at scale. Captivates integrate many underutilized physiological sensors in a streamlined package, including temple and nose temperature measurement, blink detection, head motion tracking, activity classification, 3D localization, and head pose estimation. Captivates were designed with an emphasis on: (1) manufacturing and scalability, so we can easily support large scale user studies for ourselves and offer the platform as a generalized tool for ambulatory psychophysiology research; (2) robustness and battery life, so long-term studies result in trustworthy data individual's entire day in natural environments without supervision or recharge; and (3) aesthetics and comfort, so people can wear them in their normal daily contexts without self-consciousness or changes in behavior.
Captivates are intended to enable large scale data collection without altering user behavior. We validate that our sensors capture useful data robustly for a small set of beta testers. We also show that our additional effort on aesthetics was imperative to meet our goals; namely, earlier versions of our prototype make people uncomfortable to interact naturally in public, and our additional design and miniaturization effort has made a significant impact in preserving natural behavior.
There is tremendous promise in translating psychophysiological laboratory techniques into real-world insight. Captivates serve as an open-source bridge to this end. Paired with an accurate underlying model, Captivates will be able to quantify the long-term psychological impact of our design decisions and provide real-time feedback for technologists interested in actuating a cognitively adaptive, user-aligned future.

Supplementary Material

chwalek (chwalek.zip)
Supplemental movie, appendix, image and software files for, Captivates: A Smart Eyeglass Platform for Across-Context Physiological Measurement

References

[1]
2020 (accessed November 3, 2020). Base Station. https://www.vive.com/us/accessory/base-station/
[2]
2020 (accessed November 3, 2020). Focals by North. https://www.bynorth.com/
[3]
2020 (accessed November 3, 2020). FocusEDU. https://www.brainco.tech/
[4]
2020 (accessed November 3, 2020). Meditation Made Easy. https://choosemuse.com/
[5]
2020 (accessed November 3, 2020). Open Source Tools for Neuroscience. https://openbci.com/
[6]
2020 (accessed November 3, 2020). OpenThread. https://openthread.io/
[7]
2020 (accessed November 3, 2020). Quality RTOS Embedded Software. https://www.freertos.org/
[8]
2020 (accessed November 3, 2020). Thread. https://www.threadgroup.org/
[9]
2020 (accessed November 3, 2020). Your Everyday Smart Glasses. https://vueglasses.com/
[10]
2021 (accessed April 2, 2021). The world's first wearable eyewear that lets you see yourself. https://jins-meme.com/en/
[11]
Yomna Abdelrahman, Eduardo Velloso, Tilman Dingler, Albrecht Schmidt, and Frank Vetere. 2017. Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load. IMWUT 1 (2017), 33:1-33:20.
[12]
Eugene Aidman, Carolyn Chadunow, Kayla Johnson, and John Reece. 2015. Real-time driver drowsiness feedback improves driver alertness and self-reported driving performance. Accident Analysis & Prevention 81 (2015), 8--13.
[13]
Md Zahangir Alom, Tarek M Taha, Chris Yakopcic, Stefan Westberg, Paheding Sidike, Mst Shamima Nasrin, Mahmudul Hasan, Brian C Van Essen, Abdul AS Awwal, and Vijayan K Asari. 2019. A state-of-the-art survey on deep learning theory and architectures. Electronics 8, 3 (2019), 292.
[14]
Amphenol 2018. ZTP-135SR Thermometrics Thermopile IR Sensor. Amphenol. Rev. H.
[15]
Analog Devices 2020. NDIR Thermopile-Based Gas Sensing Circuit. Analog Devices.
[16]
Celia Andreu-Sánchez, Miguel Ángel Martín-Pascual, Agnès Gruart, and José María Delgado-García. 2017. Looking at reality versus watching screens: Media professionalization effects on the spontaneous eyeblink rate. PLoS One 12, 5 (2017), e0176030.
[17]
Pietro Aricò, Gianluca Borghini, Gianluca Di Flumeri, Alfredo Colosimo, Stefano Bonelli, Alessia Golfetti, Simone Pozzi, Jean-Paul Imbert, Géraud Granger, Raïlane Benhacene, et al. 2016. Adaptive automation triggered by EEG-based mental workload index: a passive brain-computer interface application in realistic air traffic control environment. Frontiers in human neuroscience 10 (2016), 539.
[18]
Guha Balakrishnan, Fredo Durand, and John Guttag. 2013. Detecting pulse from head motions in video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3430--3437.
[19]
Tadas Baltrusaitis, Amirali Bagher Zadeh, Yao Chong Lim, and Louis-Philippe Morency. 2018. OpenFace 2.0: Facial Behavior Analysis Toolkit. 2018 13th IEEE International Conference on Automatic Face Gesture Recognition (FG 2018) (2018), 59--66.
[20]
Juan A. Castro-García, Alberto J. Molina-Cantero, Manuel Merino-Monge, and Isabel M. Gómez-González. 2019. An Open-Source Hardware Acquisition Platform for Physiological Measurements. IEEE Sensors Journal 19 (2019), 11526--11534.
[21]
Alper Cömert and Jari Hyttinen. 2015. Investigating the possible effect of electrode support structure on motion artifact in wearable bioelectric signal monitoring. BioMedical Engineering OnLine 14 (2015).
[22]
Enrico Costanza, Samuel Inverso, Elan Pavlov, Rebecca Allen, and Pattie Maes. 2006. eye-q: eyeglass peripheral display for subtle intimate notifications.
[23]
Rodney J. Croft and Roger J. Barry. 2000. Removal of ocular artifact from the EEG: a review. Neurophysiologie Clinique/Clinical Neurophysiology 30 (2000), 5--19.
[24]
Erin Cvejic, Jeesun Kim, and Chris Davis. 2010. Prosody off the top of the head: Prosodic contrasts can be discriminated by head motion. Speech Communication 52, 6 (2010), 555--564.
[25]
Ian Daly, Nicoletta Nicolaou, Slawomir Nasuto, and Kevin Warwick. 2013. Automated Artifact Removal From the Electroencephalogram: A Comparative Study. Clinical EEG and neuroscience: official journal of the EEG and Clinical Neuroscience Society (ENCS) 44 (05 2013). https://doi.org/10.1177/1550059413476485
[26]
Linh C Dang, Gregory R Samanez-Larkin, Jaime J Castrellon, Scott F Perkins, Ronald L Cowan, Paul A Newhouse, and David H Zald. 2017. Spontaneous eye blink rate (EBR) is uncorrelated with dopamine D2 receptor availability and unmodulated by dopamine agonism in healthy adults. Eneuro 4, 5 (2017).
[27]
Artem Dementyev and Christian Holz. 2017. DualBlink: A Wearable Device to Continuously Detect, Track, and Actuate Blinking For Alleviating Dry Eyes and Computer Vision Syndrome. IMWUT 1 (2017), 1:1-1:19.
[28]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[29]
Elena Di Lascio. 2018. Emotion-Aware Systems for Promoting Human Well-being. In Proceedings of the 2018 ACM International Joint Conference and 2018 International Symposium on Pervasive and Ubiquitous Computing and Wearable Computers. 529--534.
[30]
Adam Dunkels, Bjorn Gronvall, and Thiemo Voigt. 2004. Contiki-a lightweight and flexible operating system for tiny networked sensors. In 29th annual IEEE international conference on local computer networks. IEEE, 455--462.
[31]
Christoph Ellmer. 2017. OpenThread vs. Contiki IPv6: An Experimental Evaluation.
[32]
Gianluca Di Flumeri, Pietro Aricò, Gianluca Borghini, Nicolina Sciaraffa, Antonello Di Florio, and Fabio Babiloni. 2019. The Dry Revolution: Evaluation of Three Different EEG Dry Electrode Types in Terms of Signal Spectral Features, Mental States Classification and Usability. Sensors (Basel, Switzerland) 19 (2019).
[33]
Peter Gerjets, Carina Walter, Wolfgang Rosenstiel, Martin Bogdan, and Thorsten O. Zander. 2014. Cognitive state monitoring and the design of adaptive instruction in digital environments: lessons learned from cognitive workload assessment using a passive brain-computer interface approach. Frontiers in Neuroscience 8 (2014).
[34]
Gabriele Gratton. 1998. Dealing with artifacts: The EOG contamination of the event-related brain potential. Behavior Research Methods, Instruments, Computers 30 (1998), 44--53.
[35]
Kiran B. Hebbar, J. Dennis Fortenberry, Kristine Rogers, Robert Merritt, and Kirk Easley. 2005. Comparison of temporal artery thermometer to standard temperature measurements in pediatric intensive care unit patients. Pediatric critical care medicine: a journal of the Society of Critical Care Medicine and the World Federation of Pediatric Intensive and Critical Care Societies 6 5 (2005), 557--61.
[36]
Heimann Sensor 2013. Application Note NDIR-Measurement. Heimann Sensor.
[37]
Javier Hernandez, Yin Li, James M Rehg, and Rosalind W Picard. 2014. Bioglass: Physiological parameter estimation using a head-mounted wearable device. In 2014 4th International Conference on Wireless Mobile Communication and Healthcare-Transforming Healthcare Through Innovations in Mobile and Wireless Technologies (MOBIHEALTH). IEEE, 55--58.
[38]
Javier Hernandez, Rob R Morris, and Rosalind W Picard. 2011. Call center stress recognition with person-specific models. In International Conference on Affective Computing and Intelligent Interaction. Springer, 125--134.
[39]
David Hoppe, Stefan Helfmann, and Constantin A Rothkopf. 2018. Humans quickly learn to blink strategically in response to environmental task demands. Proceedings of the National Academy of Sciences 115, 9 (2018), 2246--2251.
[40]
Stephanos Ioannou, Vittorio Gallese, and Arcangelo Merla. 2014. Thermal infrared imaging in psychophysiology: Potentialities and limits. In Psychophysiology.
[41]
Shoya Ishimaru, Kai Kunze, Koichi Kise, Jens Weppner, Andreas Dengel, Paul Lukowicz, and Andreas Bulling. 2014. In the Blink of an Eye - Combining Head Motion and Eye Blink Frequency for Activity Recognition with Google Glass. ACM International Conference Proceeding Series, 150--153. https://doi.org/10.1145/2582051.2582066
[42]
Shoya Ishimaru, Kai Kunze, Koichi Kise, Jens Weppner, Andreas Dengel, Paul Lukowicz, and Andreas Bulling. 2014. In the blink of an eye: combining head motion and eye blink frequency for activity recognition with Google Glass. In AH '14.
[43]
Habib Karaki and Vasco Polyzoev. 2020 (accessed May 3, 2020). Demystifying Thermopile IR Temp Sensors. https://www.fierceelectronics.com/components/demystifying-thermopile-ir-temp-sensors-0
[44]
Matthew A Killingsworth and Daniel T Gilbert. 2010. A wandering mind is an unhappy mind. Science 330, 6006 (2010), 932--932.
[45]
Wolfgang Klimesch. 1999. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain Research Reviews 29 (1999), 169--195.
[46]
Nataliya Kosmyna, Caitlin Morris, Utkarsh Sarawgi, and Pattie Maes. 2019. AttentivU: A Biofeedback System for Real-time Monitoring and Improvement of Engagement. Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems (2019).
[47]
Saroj Lal and Ashley Craig. 2005. Reproducibility of the spectral components of the electroencephalogram during driver fatigue. International journal of psychophysiology: official journal of the International Organization of Psychophysiology 55 2 (2005), 137--43.
[48]
Celine Latulipe, Erin A Carroll, and Danielle Lottridge. 2011. Love, hate, arousal and engagement: exploring audience responses to performing arts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1845--1854.
[49]
Hayley Ledger. 2013. The effect cognitive load has on eye blinking.
[50]
Robert Makepeace and Julien Epps. 2015. Automatic task analysis based on head movement. 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2015), 5167--5170.
[51]
Tota Mizuno and Yuichiro Kume. 2015. Development of a Glasses-Like Wearable Device to Measure Nasal Skin Temperature. In HCI.
[52]
Nargess Nourbakhsh, Yuhuai Wang, and Fang Chen. 2013. GSR and Blink Features for Cognitive Load Classification. In INTERACT.
[53]
Rafal Paprocki and Artem Lenskiy. 2017. What does eye-blink rate variability dynamics tell us about cognitive performance? Frontiers in human neuroscience 11 (2017), 620.
[54]
Hae Won Park, Ishaan Grover, Samuel Spaulding, Louis Gomez, and Cynthia Breazeal. 2019. A Model-Free Affective Reinforcement Learning Approach to Personalization of an Autonomous Social Robot Companion for Early Literacy Education. In AAAI.
[55]
Francesco Pompei. [n.d.]. Temporal artery temperature detector.
[56]
PyQT. 2012. PyQt Reference Guide. (2012). http://www.riverbankcomputing.com/static/Docs/PyQt4/html/index.html
[57]
Mirko Raca, Lukasz Kidzinski, and Pierre Dillenbourg. 2015. Translating head motion into attention-towards processing of student's body-language. In Proceedings of the 8th international conference on educational data mining.
[58]
Fabian Ramseyer and Wolfgang Tschacher. 2014. Nonverbal synchrony of head-and body-movement in psychotherapy: different signals have different associations with outcome. Frontiers in psychology 5 (2014), 979.
[59]
Umair Rehman and Shi Cao. 2017. Augmented-Reality-Based Indoor Navigation: A Comparative Analysis of Handheld Devices Versus Google Glass. IEEE Transactions on Human-Machine Systems 47 (2017), 140--151.
[60]
Harry T Reis and Samuel D Gosling. 2010. Social psychological methods outside the laboratory. (2010).
[61]
Sergio Romero, Miguel Angel Mañanas, and Manel J. Barbanoj. 2008. A comparative study of automatic techniques for ocular artifact reduction in spontaneous EEG signals based on clinical target variables: A simulation case. Computers in biology and medicine 38 3 (2008), 348--60.
[62]
Soha Rostaminia, A. Mayberry, D. Ganesan, Benjamin M Marlin, and Jeremy Gummeson. 2017. iLid: Low-power Sensing of Fatigue and Drowsiness Measures on a Computational Eyeglass. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 1 2 (2017).
[63]
Phattarapong Sawangjai, Supanida Hompoonsup, Pitshaporn Leelaarporn, Supavit Kongwudhikunakorn, and Theerawit Wilaiprasitporn. 2020. Consumer Grade EEG Measuring Sensors as Research Tools: A Review. IEEE Sensors Journal 20 (2020), 3996--4024.
[64]
Zach Shelby, Klaus Hartke, and Carsten Bormann. 2014. The Constrained Application Protocol (CoAP). RFC 7252 (2014), 1--112.
[65]
Alexander Shtuchkin. 2020. DIY Position Tracking using HTC Vive's Lighthouse. https://github.com/ashtuchkin/vive-diy-position-sensor.
[66]
John A. Stern, Larry Walrath, and Rebecca Newberger Goldstein. 1984. The endogenous eyeblink. Psychophysiology 21 1 (1984), 22--33.
[67]
Rainer Stiefelhagen. 2002. Tracking focus of attention in meetings. Proceedings. Fourth IEEE International Conference on Multimodal Interfaces (2002), 273--280.
[68]
Maja Stikic, Chris Berka, Daniel J. Levendowski, Roberto F. Rubio, Veasna Tan, Stephanie Korszen, Douglas Barba, and David Wurzer. 2014. Modeling temporal sequences of cognitive state changes based on a combination of EEG-engagement, EEG-workload, and heart rate metrics. Frontiers in Neuroscience 8 (2014).
[69]
B. Tag, Andrew W. Vargo, Aman Gupta, G. Chernyshov, K. Kunze, and Tilman Dingler. 2019. Continuous Alertness Assessments: Using EOG Glasses to Unobtrusively Monitor Fatigue Levels In-The-Wild. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019).
[70]
Giacomo Tanganelli, Carlo Vallati, and Enzo Mingozzi. 2015. CoAPthon: Easy development of CoAP-based IoT applications with Python. 2015 IEEE 2nd World Forum on Internet of Things (WF-IoT) (2015), 63--68.
[71]
Texas Instruments 2011. TMP006/B Infrared Thermopile Sensor in Chip-Scale Package. Texas Instruments. Rev. C.
[72]
Chinchu Thomas and Dinesh Babu Jayagopi. 2017. Predicting student engagement in classrooms using facial behavioral cues. In Proceedings of the 1st ACM SIGCHI international workshop on multimodal interaction for education. 33--40.
[73]
Garrick L. Wallstrom, Robert E. Kass, Anita Miller, Jeffrey F. Cohn, and Nathan A. Fox. 2004. Automatic correction of ocular artifacts in the EEG: a comparison of regression-based and component-based methods. International journal of psychophysiology: official journal of the International Organization of Psychophysiology 53 2 (2004), 105--19.
[74]
Chen Wang, Erik N Geelhoed, Phil P Stenton, and Pablo Cesar. 2014. Sensing a live audience. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1909--1912.
[75]
SV Wass, K de Barbaro, and K Clackson. 2015. Tonic and phasic co-variation of peripheral arousal indices in infants. Biological Psychology 111 (2015), 26--39.
[76]
Nancy J Wei, Bryn Dougherty, Aundria Myers, and Sherif M Badawy. 2018. Using Google Glass in Surgical Settings: Systematic Review. JMIR mHealth and uHealth 6 (2018).
[77]
Fadilla Zennifa, Junko Ide, Yukio Noguchi, and Keiji Iramina. 2015. Monitoring of cognitive state on mental retardation child using EEG, ECG and NIRS in four years study. 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (2015), 6610--6613.

Cited By

View all
  • (2024)UbiHR: Resource-efficient Long-range Heart Rate Sensing on Ubiquitous DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997718:4(1-26)Online publication date: 21-Nov-2024
  • (2024)Improving Attention Using Wearables via Haptic and Multimodal Rhythmic StimuliProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642256(1-14)Online publication date: 11-May-2024
  • (2024)Democratizing EEG: Embedding Electroencephalography in a Head-Mounted Display for Ubiquitous Brain-Computer InterfacingInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2388368(1-25)Online publication date: 19-Aug-2024
  • Show More Cited By

Recommendations

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies
Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies  Volume 5, Issue 3
Sept 2021
1443 pages
EISSN:2474-9567
DOI:10.1145/3486621
Issue’s Table of Contents
This work is licensed under a Creative Commons Attribution International 4.0 License.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 September 2021
Published in IMWUT Volume 5, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. blink sensing
  2. face temperature
  3. localization
  4. robust ambulatory sensing
  5. smart eyeglass

Qualifiers

  • Research-article
  • Research
  • Refereed

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)328
  • Downloads (Last 6 weeks)38
Reflects downloads up to 24 Dec 2024

Other Metrics

Citations

Cited By

View all
  • (2024)UbiHR: Resource-efficient Long-range Heart Rate Sensing on Ubiquitous DevicesProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36997718:4(1-26)Online publication date: 21-Nov-2024
  • (2024)Improving Attention Using Wearables via Haptic and Multimodal Rhythmic StimuliProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642256(1-14)Online publication date: 11-May-2024
  • (2024)Democratizing EEG: Embedding Electroencephalography in a Head-Mounted Display for Ubiquitous Brain-Computer InterfacingInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2388368(1-25)Online publication date: 19-Aug-2024
  • (2023)AirSpec: A Smart Glasses Platform, Tailored for Research in the Built EnvironmentAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610796(204-204)Online publication date: 8-Oct-2023
  • (2023)NugiTex: An Interactive, Affective Wearable that Informs Users of a Plant’s “Comfort” Level through Haptic CuesCompanion Publication of the 2023 ACM Designing Interactive Systems Conference10.1145/3563703.3596651(251-255)Online publication date: 10-Jul-2023
  • (2023)Affective Computing for detecting psychological Flow state: a definition and methodological problem2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)10.1109/ACIIW59127.2023.10388109(1-5)Online publication date: 10-Sep-2023
  • (2023)Artificial intelligence enabled smart digital eye wearablesCurrent Opinion in Ophthalmology10.1097/ICU.000000000000098534:5(414-421)Online publication date: 28-Jun-2023
  • (2023)3D Printing Smart Eyeglass Frames: A ReviewACS Applied Engineering Materials10.1021/acsaenm.2c002651:4(1142-1163)Online publication date: 7-Mar-2023
  • (2022)EquinoxProceedings of the 1st ACM Workshop on Smart Wearable Systems and Applications10.1145/3556560.3560713(1-6)Online publication date: 17-Oct-2022
  • (2022)LiSeeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35502826:3(1-30)Online publication date: 7-Sep-2022
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Full Access

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media