More Web Proxy on the site http://driver.im/

research-article

Public Access

Cyber-physical Support of Daily Activities: A Robot/Smart Home Partnership

Authors:

Christopher Pereyda,

Nisha Raghunath,

Garrett Wilson,

Maureen Schmitter-Edgecombe,

Diane J. CookAuthors Info & Claims

ACM Transactions on Cyber-Physical Systems, Volume 4, Issue 2

Article No.: 21, Pages 1 - 24

https://doi.org/10.1145/3365225

Published: 12 December 2019 Publication History

All formats PDF

Abstract

This article introduces RAS, a cyber-physical system that supports individuals with memory limitations to perform daily activities in their own homes. RAS represents a partnership between a smart home, a robot, and software agents. When smart home residents perform activities, RAS senses their movement in the space and identifies the current activity. RAS tracks activity steps to detect omission errors. When an error is detected, the RAS robot finds and approaches the human with an offer of assistance. Assistance consists of playing a video recording of the entire activity, showing the omitted activity step, or guiding the resident to the object that is required for the current step. We evaluated RAS performance for 54 participants performing three scripted activities in a smart home testbed and for 2 participants using the system over multiple days in their own homes. In the testbed experiment, activity errors were detected with a sensitivity of 0.955 and specificity of 0.992. RAS assistance was performed successfully with a rate of 0.600. In the in-home experiments, activity errors were detected with a combined sensitivity of 0.905 and a combined specificity of 0.988. RAS assistance was performed successfully for the in-home experiments with a rate of 0.830.

References

[1]

Future Market Insights. 2018. Cyber-physical system market: Global industry analysis (2013-2017) and opportunity assessment (2018--2028). https://www.futuremarketinsights.com/reports/cyber-physical-systems-market.

[2]

Web of Science. 2019. Web of Science: Search=cyber-physical systems; sort=year. 2019.

[3]

J. Iriondo and J. Jordan. 2018. Older people projected to outnumber children for first time in U. S. history, 2018. https://www.census.gov/newsroom/press-releases/2018/cb18-41-population-projections.html.

[4]

P. Buonocunto, A. Giantomassi, M. Marinoni, D. Calvaresi, and G. Buttazzo. 2018. A limb tracking platform for tele-rehabilitation. ACM Trans. Cyber Phys. Syst. 2, 4 (2018), 30 2018.

Digital Library

[5]

J. A. Ramirez-Bautista and J. A. Huerta-Ruelas. 2017. A review in detection and monitoring gait disorders using in-shoe plantar measurement systems. IEEE Rev. Biomed. Eng. 10, 299--309, 2017.

[6]

M. Mancini et al. 2016. Continuous monitoring of turning mobility and its association to falls and cognitive function: A pilot study. J. Gerontol. Ser. A Biol. Sci. Med. Sci. 71, 8 (2016), 1102--1108, 2016.

[7]

T. L. Hayes, T. Riley, N. Mattek, M. Pavel, and J. A. Kaye. 2014. Sleep habits in mild cognitive impairment. Alzheimer Dis. Assoc. Disord. 28 (2014), 145--150.

[8]

G. Sprint, D. Cook, R. Fritz, and M. Schmitter-Edgecombe. 2016. Detecting health and behavior change by analyzing smart home sensor data. In Proceedings of the IEEE International Conference on Smart Computing (SMARTCOMP’16).

[9]

S. Difrancesco et al. 2016. Out-of-home activity recognition from GPS data in schizophrenic patients. In Proceedings of the IEEE International Symposium on Computer-based Medical Systems. 324--328.

[10]

M. Boukhechba, Y. Huang, P. Chow, K. Fua, B. A. Teachman, and L. E. Barnes. 2017. Monitoring social anxiety from mobility and communication patterns. Proceedings of the ACM International Joint Conference on Pervasive Ubiquitous Computing. 749--753.

[11]

J. C. Quiroz, M. H. Yong, and E. Geangu. 2017. Emotion-recognition using smart watch accelerometer data: Preliminary findings. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 805--812.

[12]

J. Petersen, D. Austin, N. Mattek, and J. Kaye. 2015. Time out-of-home and cognitive, physical, and emotional wellbeing of older adults: A longitudinal mixed effects model. PLoS One 10, 10, Article e0139643 (2015).

[13]

D. J. Cook, M. Schmitter-Edgecombe, L. Jonsson, and A. V. Morant. 2018. Technology-enabled assessment of functional health. IEEE Rev. Biomed. Eng. 12 (2018), 319--332.

[14]

S. Majumder et al. 2017. Smart homes for elderly healthcare—Recent advances and research challenges. Sensors, 17, 11 (2017) 2496

[15]

A. Pratap, J. A. Anguera, and B. N. Renn. 2017. The feasibility of using smartphones to assess and remediate depression in Hispanic/Latino individuals nationally. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 854--860.

[16]

J. Austin, H. H. Dodge, T. Riley, P. G. Jacobs, S. Thielke, and J. Kaye. 2016. A smart-home system to unobtrusively and continuously assess loneliness in older adults. IEEE J. Transl. Eng. Heal. Med. 4, Article 7488979 (2016).

[17]

T. Grover and G. Mark. 2017. Digital footprints: Predicting personality from temporal patterns of technology use. In Proceedings of the ACM International Joint Conference on Pervasive and Ubiquitous Computing. 41--44.

[18]

G. Sprint, D. Cook, D. Weeks, and V. Borisov. 2015. Predicting functional independence measure scores during rehabilitation with wearable inertial sensors. IEEE Access 3:1350--1366.

[19]

J. D. Howcroft, E. D. Lemaire, J. Kofman, and W. E. McIlroy. 2014. Analysis of dual-task elderly gait using wearable plantar-pressure insoles and accelerometer. In Proceedings of the International Conference of the IEEE Engineering in Medicine and Biology Society. 5003--5006.

[20]

M. Cella, L. Okruszek, M. Lawrence, V. Zarlenga, Z. He, and T. Wykes. 2017. Using wearable technology to detect the autonomic signature of illness severity in schizophrenia. Schizophr. Res. 195 (2017), 537--542.

[21]

D. J. Cook, G. Sprint, R. Fritz, and G. E. Duncan. 2018. Using smart city technology to make healthcare smarter. Proc. IEEE 106, 4 (2018), 708--722.

[22]

J. Wang, Z. Liu, and Y. Wu. 2013. Learning actionlet ensemble for 3D human action recognition. IEEE Trans. Pattern Anal. Mach. Intell. 36, 5 (2013), 914--927.

[23]

M. Mancini et al. 2016. Continuous monitoring of turning mobility and its association to falls and cognitive function: A pilot study. J. Gerontol. Ser. A Biol. Sci. Med. Sci. 71, 8 (2016), 1102--1108.

[24]

A. Ejupi, M. Brodie, S. R. Lord, J. Annegam, S. J. Redmond, and K. Delbaere. 2017. Wavelet-based sit-to-stand detection and assessment of fall risk in older people using a wearable pendant device. IEEE Trans. Biomed. Eng. 64, 7 (2017), 1602--1607.

[25]

D. J. Cook, P. Dawadi, and M. Schmitter-Edgecombe. 2015. Analyzing activity behavior and movement in a naturalistic environment using smart home techniques. IEEE J. Biomed. Heal. Inform. 19, 6 (2015), 1882--1892.

[26]

M. A. Ul Alam, N. Roy, S. Holmes, A. Gangopadhyay, and E. Galik. 2016. Automated functional and behavioral health assessment of older adults with dementia. In Proceedings of the International Conference on Connected Health: Applications, Systems and Engineering Technologies. 140--149.

[27]

S. Robben, G. Englebienne, and B. Krose. 2017. Delta features from ambient sensor data are good predictors of change in functional health. IEEE J. Biomed. Heal. Inform. 21, 4 (2017), 986--993.

[28]

A. Akl, B. Taati, and A. Mihailidis. 2015. Autonomous unobtrusive detection of mild cognitive impairment in older adults. IEEE Trans. Biomed. Eng. 62, 5 (2015), 1383--1394.

[29]

N. Dey, A. S. Ashour, F. Shi, S. J. Fong, and J. M. R. S. Tavares. 2018. Medical cyber-physical systems: A survey. J. Med. Syst. 42, 74 (2018).

Digital Library

[30]

Y. Zhang, M. Qiu, C.-W. Tsai, M. M. Hassan, and A. Alamri. 2015. Health-CPS: Healthcare cyber-physical system assisted by cloud and big data. IEEE Syst. J. 11, 1 (2015), 88--95.

[31]

H. M. Do, M. Pham, W. Sheng, D. Yang, and M. Liu. 2018. RiSH: A robot-integrated smart home for elderly care. Rob. Auton. Syst 101 (2018), 74--92.

[32]

R. H. Wang, A. Sudhama, M. Begum, and R. Huq. 2017. Robots to assist daily activities: Views of older adults with Alzheimer's disease and their caregivers. Int. Psychogeriatrics 29, 1 (2017), 67--79.

[33]

M. Begum, R. Huq, R. H. Wang, and A. Mihailidis. 2015. Collaboration of an assistive robot and older adults with dementia. Gerontechnology 13, 4 (2015), 405--419.

[34]

T. L. Mitzner et al. 2010. Older adults talk technology: Technology usage and attitudes. Comput. Human Behav. 26, 6 (2010), 1710--1721.

Digital Library

[35]

G. Hoffman and W. Ju. 2014. Designing robots with movement in mind. J. Human-Robot Interact. 3, 1 (2014), 91--122.

Digital Library

[36]

M. Destephe, T. Maruyama, M. Zecca, K. Hashimoto, and A. Takanishi. 2013. Improving the human-robot interaction through emotive movements A special case: Walking. Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction.

[37]

A. Bisio et al. 2014. Motor contagion during human-human and human-robot interaction. PLoS One 9, 8 (2014), e106172.

[38]

A. Kupferberg, S. Glasauer, M. Huber, M. Rickert, A. Knoll, and T. Brandt. 2011. Biological movement increases acceptance of humanoid robots as human partners in motor interaction. AI Soc. 26, 4 (2011), 339--345.

Digital Library

[39]

A. H. Qureshi, Y. Nakamura, Y. Yoshikawa, and H. Ishiguro. 2016. Robot gains social intelligence through multimodal deep reinforcement learning. In Proceedings of the IEEE-RAS International Conference on Humanoid Robots. 745--751.

[40]

Honda. 2007. Technical information. Asimo, 2007. Retrieved from http://asimo.honda.com/downloads/pdf/asimo-technical-information.pdf.

[41]

M. J. Mataric. 2017. Socially assistive robotics: Human augmentation versus automation. Sci. Robot. 2 (2017), 1--3.

[42]

K. M. Goher, N. Mansouri, and S. O. Fadlallah. 2017. Assessment of personal care and medical robots from older adults’ perspective. Robot. Biomimetics 4, 1 (2017) 5.

[43]

Y.-T. Liao, C.-H. Zong, H.-H. Lee, and E. Tanaka. 2018. Development of kinect-based upper-limb assistance device for the motions of activities of daily living. In Proceedings of the IEEE International Conference on Cyborg and Bionic Systems.

[44]

Riken. 2015. The strong robot with the gentle touch. News and Media, 2015. Retrieved from http://www.riken.jp/en/pr/press/2015/20150223_2/.

[45]

J. Fasola and M. J. Mataric. 2013. A socially assistive robot exercise coach for the elderly. J. Human-Robot Interact. 2, 2 (2013), 3--32.

Digital Library

[46]

L. Pu, W. Moyle, C. Jones, and M. Todorovic. 2019. The effectiveness of social robots for older adults: A systematic review and meta-analysis of randomized controlled studies. Gerontologist 59, 1 (2019), e37--e51.

[47]

K. Wada, T. Shibata, T. Asada, and T. Musha. 2007. Robot therapy for prevention of dementia at home. J. Robot. Mech. 19, 6 (2007), 691.

[48]

P. Bovbel and G. Nejat. 2014. Casper: An assistive kitchen robot to promote aging in place. J. Med. Device. 8, 3 (2014).

[49]

J. Hoey, A. Von Bertoldi, T. Craig, P. Poupart, and A. Mihailidis. 2010. Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process. Comput. Vis. Image Underst. 114, 5 (2010), 503--519.

Digital Library

[50]

J. Hoey, T. Plotz, D. Jackson, A. Monk, C. Pham, and P. Olivier. 2011. Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive Mob. Comput. 7, 3 (2011), 299--318.

Digital Library

[51]

T. Kosch, P. W. Wozniak, E. Brady, and A. Schmidt. 2018. Smart kitchens for people with cognitive impairments: A qualitative study of design requirements. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 271.

[52]

S. Yan, K.-J. Lin, X. Zheng, and W. Zhang. 2019. Using latent knowledge to improve real-time activity recognition for smart IoT. IEEE Trans. Knowl. Data Eng. 2019.

[53]

J. Wang, Y. Chen, S. Hao, X. Peng, and L. Hu. 2019. Deep learning for sensor-based activity recognition: A survey. Pattern Recog. Lett. 119 (2019), 3--11.

Digital Library

[54]

B. Ghanem et al. 2018. ActivityNet large-scale activity recognition challenge 2018. Retrieved from http://activity-net.org/challenges/2018/index.html.

[55]

J. A. Stankovic. 2017. Research directions for cyber physical systems in wireless and mobile healthcare. ACM Trans. Cyber Phys. Syst. 1, 1 (2017), 1.

Digital Library

[56]

B. Das, D. J. Cook, N. Krishnan, and M. Schmitter-Edgecombe. 2016. One-class classification-based real-time activity error detection in smart homes. IEEE J. Sel. Top. Signal Process. 10, 5 (2016), 914--923.

[57]

M. F. Schwartz, L. J. Buxbaum, M. W. Montgomery, E. Fitzpatrick-DeSalme, T. Hart, and M. Ferraro. 1999. Naturalistic action production following right hemisphere stroke. Neuropsychologia, 37 (1999), 51--66.

[58]

M. M. N. Bienkiewicz, M.-L. Brandi, G. Goldenberg, C. M. L. Hughes, and J. Hermsdorfer. 2014. The tool in the brain: Apraxia in ADL. Behavioral and neurological correlates of apraxia in daily living. Front. Psychol. 5 (2014), 353.

[59]

S. Thrun, W. Burgard, and D. Fox. 2005. Probabilistic Robotics. The MIT Press.

[60]

G. Cheng, J. Han, P. Zhou, and D. Xu. 2019. Learning rotation-invariant and Fisher discriminative convolutional neural networks for object detection. IEEE Trans. Image Proc. 28, 1 (2019), 265--278.

Digital Library

[61]

J. Huang et al. 2016. Speed/accuracy trade-offs for modern convolutional object detectors. Retrieved from CoRR, abs/1611.1, 2016.

[62]

B. Reisburg et al. 2001. The Alzheimer's disease activities of daily living international scale (ADL-IS). Int. Psychogeriatrics 13, 2 (2001), 163--181.

[63]

M. Diel, M. Marsiske, A. Horgas, A. Rosenberg, J. Saczynski, and S. Willis. 2005. The revised observed tasks of daily living: A performance-based assessment of everyday problem solving in older adults. J. Appl. Gerontechnology 24 (2005), 211--230.

[64]

M. F. Folstein, S. E. Folstein, and P. R. McHugh. 1975. Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J. Psych. Res. 12, 4 (1975), 189--198.

[65]

Estimote. 2018. The physical world. Software-defined, 2018. Retrieved from https://estimote.com/.

[66]

G. Wilson et al. 2019. Robot-enabled support of daily activities in smart home environments. Cogn. Syst. Res. 54 (2019), 258--272.

Cited By

Wang SKo RBai GDong NChoi TZhang Y(2024)Evasion Attack and Defense on Machine Learning Models in Cyber-Physical Systems: A SurveyIEEE Communications Surveys & Tutorials10.1109/COMST.2023.334480826:2(930-966)Online publication date: Oct-2025
https://doi.org/10.1109/COMST.2023.3344808
Owens OBeer JLeonard MSudduth BBurton LWu X(2024)(Re)Defining Smart Home Through an HCI Perspective: A Systematic Review of over Two Decades of Smart Home Conceptualization and ResearchInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2437112(1-24)Online publication date: 13-Dec-2024
https://doi.org/10.1080/10447318.2024.2437112
Duan PYu ZGao KMeng LHan YYe F(2024)Solving the multi-objective path planning problem for mobile robot using an improved NSGA-II algorithmSwarm and Evolutionary Computation10.1016/j.swevo.2024.10157687(101576)Online publication date: Jun-2024
https://doi.org/10.1016/j.swevo.2024.101576
Show More Cited By

Index Terms

Cyber-physical Support of Daily Activities: A Robot/Smart Home Partnership

Recommendations

Recognizing water-based activities in the home through infrastructure-mediated sensing
UbiComp '12: Proceedings of the 2012 ACM Conference on Ubiquitous Computing

Activity recognition in the home has been long recognized as the foundation for many desirable applications in fields such as home automation, sustainability, and healthcare. However, building a practical home activity monitoring system remains a ...
Annotating smart environment sensor data for activity learning
Smart Environments: Technology to Support Healthcare

The pervasive sensing technologies found in smart homes offer unprecedented opportunities for providing health monitoring and assistance to individuals experiencing difficulties living independently at home. In order to monitor the functional health of ...
Recognising activities of daily living from patterns of object use

To provide assistance and support to the elderly disabled and cognitively impaired, the recognition of their activities of daily living (ADL) must be accurate and precise with regards to the object use for the activity situations. Current ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Transactions on Cyber-Physical Systems

ACM Transactions on Cyber-Physical Systems Volume 4, Issue 2

April 2020

266 pages

ISSN:2378-962X

EISSN:2378-9638

DOI:10.1145/3372402

Editor:
Tei-Wei Kuo
City University of Hong Kong and National Taiwan University, Taiwan

Issue’s Table of Contents

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Journal Family

ACM Journals for the Design of Smart and Connected Systems

Publication History

Published: 12 December 2019

Accepted: 01 August 2019

Revised: 01 August 2019

Received: 01 May 2019

Published in TCPS Volume 4, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Science Foundation
National Institutes of Health

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
884
Total Downloads

Downloads (Last 12 months)139
Downloads (Last 6 weeks)24

Reflects downloads up to 21 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Wang SKo RBai GDong NChoi TZhang Y(2024)Evasion Attack and Defense on Machine Learning Models in Cyber-Physical Systems: A SurveyIEEE Communications Surveys & Tutorials10.1109/COMST.2023.334480826:2(930-966)Online publication date: Oct-2025
https://doi.org/10.1109/COMST.2023.3344808
Owens OBeer JLeonard MSudduth BBurton LWu X(2024)(Re)Defining Smart Home Through an HCI Perspective: A Systematic Review of over Two Decades of Smart Home Conceptualization and ResearchInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2437112(1-24)Online publication date: 13-Dec-2024
https://doi.org/10.1080/10447318.2024.2437112
Duan PYu ZGao KMeng LHan YYe F(2024)Solving the multi-objective path planning problem for mobile robot using an improved NSGA-II algorithmSwarm and Evolutionary Computation10.1016/j.swevo.2024.10157687(101576)Online publication date: Jun-2024
https://doi.org/10.1016/j.swevo.2024.101576
Sandhu MSilvera-Tawil DBorges PZhang QKusy B(2024)Internet of robotic things for independent living: Critical analysis and future directionsInternet of Things10.1016/j.iot.2024.10112025(101120)Online publication date: Apr-2024
https://doi.org/10.1016/j.iot.2024.101120
Miura CChen SSaiki SNakamura MYasuda K(2022)Assisting Personalized Healthcare of Elderly People: Developing a Rule-Based Virtual Caregiver System Using Mobile ChatbotSensors10.3390/s2210382922:10(3829)Online publication date: 18-May-2022
https://doi.org/10.3390/s22103829
Ganesan APaul ASeo H(2022)Elderly People Activity Recognition in Smart Grid Monitoring EnvironmentMathematical Problems in Engineering10.1155/2022/95400332022(1-12)Online publication date: 22-Mar-2022
https://doi.org/10.1155/2022/9540033
Wang GWen HZhou J(2022)Indoor Positioning Technology and Control of Mobile Home Service RobotScientific Programming10.1155/2022/12683932022Online publication date: 1-Jan-2022
https://dl.acm.org/doi/10.1155/2022/1268393
Son HLee CFakhrHosseini SLee SCoughlin JRudnik J(2022)Reshaping the Smart Home Research and Development in the Pandemic Era: Considerations around Scalable and Easy-to-Install DesignProceedings of the ACM on Human-Computer Interaction10.1145/35129616:CSCW1(1-28)Online publication date: 7-Apr-2022
https://dl.acm.org/doi/10.1145/3512961
Khodabandehloo EAlimohammadi ARiboni D(2022)FreeSia: A Cyber-physical System for Cognitive Assessment through Frequency-domain Indoor Locomotion AnalysisACM Transactions on Cyber-Physical Systems10.1145/34704546:2(1-31)Online publication date: 11-Apr-2022
https://dl.acm.org/doi/10.1145/3470454
Lachaux KMaitre JBouchard KLussier MBottari CCouture MBier NGiroux SGaboury S(2022)Managing the Fall Risk in Smart Homes Using Monocular Cameras, Ambient Sensors and an Interactive Social Robot2022 IEEE International Conference on Blockchain, Smart Healthcare and Emerging Technologies (SmartBlock4Health)10.1109/SmartBlock4Health56071.2022.10034526(1-10)Online publication date: 24-Oct-2022
https://doi.org/10.1109/SmartBlock4Health56071.2022.10034526
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Media

Figures

Other

Tables

View Issue’s Table of Contents