More Web Proxy on the site http://driver.im/

research-article

NuActiv: recognizing unseen new activities using semantic attribute-based learning

Authors:

Heng-Tze Cheng,

Di YouAuthors Info & Claims

MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and services

Pages 361 - 374

https://doi.org/10.1145/2462456.2464438

Published: 25 June 2013 Publication History

Abstract

We study the problem of how to recognize a new human activity when we have never seen any training example of that activity before. Recognizing human activities is an essential element for user-centric and context-aware applications. Previous studies showed promising results using various machine learning algorithms. However, most existing methods can only recognize the activities that were previously seen in the training data. A previously unseen activity class cannot be recognized if there were no training samples in the dataset. Even if all of the activities can be enumerated in advance, labeled samples are often time consuming and expensive to get, as they require huge effort from human annotators or experts. In this paper, we present NuActiv, an activity recognition system that can recognize a human activity even when there are no training data for that activity class. Firstly, we designed a new representation of activities using semantic attributes, where each attribute is a human readable term that describes a basic element or an inherent characteristic of an activity. Secondly, based on this representation, a two-layer zero-shot learning algorithm is developed for activity recognition. Finally, to reinforce recognition accuracy using minimal user feedback, we developed an active learning algorithm for activity recognition. Our approach is evaluated on two datasets, including a 10-exercise-activity dataset we collected, and a public dataset of 34 daily life activities. Experimental results show that using semantic attribute-based learning, NuActiv can generalize knowledge to recognize unseen new activities. Our approach achieved up to 79% accuracy in unseen activity recognition.

References

[1]

M. Azizyan, I. Constandache, and R. Roy Choudhury. SurroundSense: Mobile phone localization via ambience fingerprinting. In Proc. Int'l Conf. Mobile Computing and Networking, pages 261--272, 2009.

Digital Library

[2]

L. Bao and S. S. Intille. Activity recognition from user-annotated acceleration data. In Proceedings of The International Conference on Pervasive Computing, pages 1--17. Springer, 2004.

[3]

E. Berlin and K. Van Laerhoven. Detecting leisure activities with dense motif discovery. In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, UbiComp '12, pages 250--259, New York, NY, USA, 2012. ACM.

Digital Library

[4]

C. M. Bishop. Pattern Recognition and Machine Learning. Springer-Verlag Inc., 2006.

Digital Library

[5]

U. Blanke and B. Schiele. Remember and transfer what you have learned - recognizing composite activities based on activity spotting. In International Symposium on Wearable Computers (ISWC), pages 1--8, Oct. 2010.

[6]

C.-C. Chang and C.-J. Lin. LIBSVM: a library for support vector machines, 2001. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm.

Digital Library

[7]

K.-H. Chang, M. Y. Chen, and J. Canny. Tracking free-weight exercises. In Proc. Int'l Conf. Ubiquitous computing, UbiComp '07, pages 19--37, 2007.

Digital Library

[8]

Google Inc. Google Glass, Apr. 2013. http://www.google.com/glass/.

[9]

T. Huynh, M. Fritz, and B. Schiele. Discovery of activity patterns using topic models. In Proceedings of The International Conference on Ubiquitous Computing, UbiComp '08, pages 10--19, New York, NY, USA, 2008. ACM.

Digital Library

[10]

S. Kang, J. Lee, H. Jang, H. Lee, Y. Lee, S. Park, T. Park, and J. Song. Seemon: scalable and energy-efficient context monitoring framework for sensor-rich mobile environments. In Proceedings of The 6th International Conference On Mobile Systems, Applications, And Services, MobiSys '08, pages 267--280, New York, NY, USA, 2008. ACM.

Digital Library

[11]

S. Kang, Y. Lee, C. Min, Y. Ju, T. Park, J. Lee, Y. Rhee, and J. Song. Orchestrator: An active resource orchestration framework for mobile context monitoring in sensor-rich mobile environments. In International Conference on Pervasive Computing and Communications (PerCom), pages 135--144, 2010.

[12]

C. Kemp, J. B. Tenenbaum, T. L. Griffiths, T. Yamada, and N. Ueda. Learning systems of concepts with an in finite relational model. In Proceedings of the 21st national conference on Artificial intelligence - Volume 1, AAAI'06, pages 381--388. AAAI Press, 2006.

Digital Library

[13]

C. H. Lampert, H. Nickisch, and S. Harmeling. Learning to detect unseen object classes by between-class attribute transfer. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 951--958, 2009.

[14]

N. D. Lane, E. Miluzzo, H. Lu, D. Peebles, T. Choudhury, and A. T. Campbell. A survey of mobile phone sensing. IEEE Comm. Magazine, 48:140--150, September 2010.

Digital Library

[15]

H. Larochelle, D. Erhan, and Y. Bengio. Zero-data learning of new tasks. In Proc. Conf. on Artificial intelligence, AAAI'08, pages 646--651, 2008.

Digital Library

[16]

J. Liu, B. Kuipers, and S. Savarese. Recognizing human actions by attributes. In Conf. Computer Vision and Pattern Recognition, pages 3337--3344, 2011.

Digital Library

[17]

R. Liu, T. Chen, and L. Huang. Research on human activity recognition based on active learning. In International Conference on Machine Learning and Cybernetics (ICMLC), volume 1, pages 285--290, 2010.

[18]

B. Logan, J. Healey, M. Philipose, E. M. Tapia, and S. Intille. A long-term evaluation of sensing modalities for activity recognition. In Proc. Int'l Conf. Ubiquitous computing, UbiComp '07, pages 483--500, 2007.

Digital Library

[19]

B. Longstaff, S. Reddy, and D. Estrin. Improving activity classification for health applications on mobile devices using active and semi-supervised learning. In International Conference on Pervasive Computing Technologies for Healthcare, PervasiveHealth'10, pages 1--7, 2010.

[20]

H. Lu, W. Pan, N. Lane, T. Choudhury, and A. Campbell. SoundSense: Scalable sound sensing for people-centric applications on mobile phones. In Proc. Int'l Conf. Mobile systems, applications, and services, pages 165--178, 2009.

Digital Library

[21]

H. Lu, J. Yang, Z. Liu, N. D. Lane, T. Choudhury, and A. T. Campbell. The Jigsaw continuous sensing engine for mobile phone applications. In Proc. ACM Conf. Embedded Networked Sensor Systems, SenSys '10, pages 71--84, 2010.

Digital Library

[22]

M. Mahdaviani and T. Choudhury. Fast and scalable training of semi-supervised CRFs with application to activity recognition. In J. C. Platt, D. Koller, Y. Singer, and S. Roweis, editors, Advances in Neural Information Processing Systems, pages 977--984. MIT Press, Cambridge, MA, 2007.

[23]

L. M. Manevitz and M. Yousef. One-class SVMs for document classification. Journal of Machine Learning Research, 2:139--154, Mar. 2002.

Digital Library

[24]

E. Miluzzo, C. T. Cornelius, A. Ramaswamy, T. Choudhury, Z. Liu, and A. T. Campbell. Darwin phones: the evolution of sensing and inference on mobile phones. In Proc. Int'l Conf. Mobile systems, applications, and services, pages 5--20, 2010.

Digital Library

[25]

E. Miluzzo, N. D. Lane, K. Fodor, R. Peterson, H. Lu, M. Musolesi, S. B. Eisenman, X. Zheng, and A. T. Campbell. Sensing meets mobile social networks: the design, implementation and evaluation of the CenceMe application. In Proc. ACM Conf. Embedded network sensor systems, SenSys '08, pages 337--350, 2008.

Digital Library

[26]

D. Minnen, T. Starner, I. Essa, and C. Isbell. Discovering characteristic actions from on-body sensor data. In In Proc. International Symposium On Wearable Computing, pages 11--18, 2006.

[27]

Motorola Mobility. MotoACTV, Apr. 2013. https://motoactv.com/.

[28]

M. Muehlbauer, G. Bahle, and P. Lukowicz. What can an arm holster worn smart phone do for activity recognition? In Proc. International Symposium on Wearable Computers, ISWC '11, pages 79--82, 2011.

Digital Library

[29]

S. Nath. ACE: exploiting correlation for energy-efficient and continuous context sensing. In Proc. Int'l Conf. Mobile systems, applications, and services, MobiSys '12, pages 29--42, 2012.

Digital Library

[30]

M. Palatucci, D. Pomerleau, G. E. Hinton, and T. M. Mitchell. Zero-shot learning with semantic output codes. In Proceedings of the Neural Information Processing Systems (NIPS), pages 1410--1418, 2009.

[31]

D. Parikh and K. Grauman. Interactively building a discriminative vocabulary of nameable attributes. In Proc. Int'l Conf. Computer Vision and Pattern Recognition, pages 1681--1688, 2011.

Digital Library

[32]

D. Parikh and K. Grauman. Relative attributes. In IEEE International Conference on Computer Vision (ICCV), pages 503--510, 2011.

Digital Library

[33]

S. Rosenthal, A. K. Dey, and M. Veloso. Using decision-theoretic experience sampling to build personalized mobile phone interruption models. In Proc. International Conference on Pervasive Computing, Pervasive'11, pages 170--187, Berlin, Heidelberg, 2011. Springer-Verlag.

Digital Library

[34]

O. Russakovsky and L. Fei-Fei. Attribute learning in large-scale datasets. In Proceedings of the 11th European Conference on Trends and Topics in Computer Vision, ECCV'10, pages 1--14, Berlin, Heidelberg, 2012. Springer-Verlag.

Digital Library

[35]

B. Settles. Active Learning. Morgan & Claypool, 2012.

[36]

M. Stikic, D. Larlus, S. Ebert, and B. Schiele. Weakly supervised recognition of daily life activities with wearable sensors. IEEE Trans. Pattern Analysis and Machine Intelligence, 33(12):2521--2537, 2011.

Digital Library

[37]

M. Stikic, K. Van Laerhoven, and B. Schiele. Exploring semi-supervised and active learning for activity recognition. In Proc. International Symposium on Wearable Computers (ISWC), pages 81--88, 2008.

Digital Library

[38]

U.S. Bureau of Labor Statistics. American time use survey activity lexicon. American Time Use Survey, 2011.

[39]

S. Wang, W. Pentney, A.-M. Popescu, T. Choudhury, and M. Philipose. Common sense based joint training of human activity recognizers. In Proceedings of the 20th international joint conference on Artifical intelligence, IJCAI'07, pages 2237--2242, San Francisco, CA, USA, 2007. Morgan Kaufmann Publishers Inc.

Digital Library

[40]

Y. Wang, J. Lin, M. Annavaram, Q. A. Jacobson, J. Hong, B. Krishnamachari, and N. Sadeh. A framework of energy efficient mobile sensing for automatic user state recognition. In Proc. Int'l Conf. Mobile systems, applications, and services, MobiSys '09, pages 179--192, 2009.

Digital Library

[41]

V. W. Zheng, D. H. Hu, and Q. Yang. Cross-domain activity recognition. In Proc. International Conference on Ubiquitous Computing, Ubicomp '09, pages 61--70, New York, NY, USA, 2009. ACM.

Digital Library

Cited By

Ye XSakurai KNair NWang K(2024)Machine Learning Techniques for Sensor-Based Human Activity Recognition with Data Heterogeneity—A ReviewSensors10.3390/s2424797524:24(7975)Online publication date: 13-Dec-2024
https://doi.org/10.3390/s24247975
Le HLakshminarayanan RLi JMishra VIntille S(2024)Collecting Self-reported Physical Activity and Posture Data Using Audio-based Ecological Momentary AssessmentProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785848:3(1-35)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678584
Yin YXie LJiang ZXiao FCao JLu S(2024)A Systematic Review of Human Activity Recognition Based on Mobile Devices: Overview, Progress and TrendsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.335759126:2(890-929)Online publication date: Oct-2025
https://doi.org/10.1109/COMST.2024.3357591
Show More Cited By

Index Terms

NuActiv: recognizing unseen new activities using semantic attribute-based learning
1. Computer systems organization
  1. Embedded and cyber-physical systems
  2. Real-time systems
2. Computing methodologies
  1. Machine learning
    1. Learning paradigms
      1. Supervised learning
        Supervised learning by classification
    2. Machine learning approaches
      1. Classification and regression trees

Recommendations

Towards zero-shot learning for human activity recognition using semantic attribute sequence model
UbiComp '13: Proceedings of the 2013 ACM international joint conference on Pervasive and ubiquitous computing

Understanding human activities is important for user-centric and context-aware applications. Previous studies showed promising results using various machine learning algorithms. However, most existing methods can only recognize the activities that were ...
Non-obstructive room-level locating system in home environments using activity fingerprints from smartwatch
UbiComp '15: Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing

Many smart home applications, such as monitoring for the elderly and home automation, require location information for individual occupants. Several techniques have been proposed for tracking occupants in a home environment. However, the current ...
Aerobic activity monitoring: towards a long-term approach

With recent progress in wearable sensing, it becomes reasonable for individuals to wear different sensors all day, and thus, global activity monitoring is establishing. The goals in global activity monitoring systems are among others to tell the type of ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MobiSys '13: Proceeding of the 11th annual international conference on Mobile systems, applications, and services

June 2013

568 pages

ISBN:9781450316729

DOI:10.1145/2462456

General Chairs:
Hao-Hua Chu
National Taiwan University
,
Polly Huang
National Taiwan University
,
Program Chairs:
Romit Roy Choudhury
Duke University
,
Feng Zhao
Microsoft Research

Copyright © 2013 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMOBILE: ACM Special Interest Group on Mobility of Systems, Users, Data and Computing

In-Cooperation

SIGOPS: ACM Special Interest Group on Operating Systems

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 June 2013

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MobiSys'13

Sponsor:

SIGMOBILE

MobiSys'13: The 11th Annual International Conference on Mobile Systems, Applications, and Services

June 25 - 28, 2013

Taipei, Taiwan

Acceptance Rates

MobiSys '13 Paper Acceptance Rate 33 of 211 submissions, 16%;

Overall Acceptance Rate 274 of 1,679 submissions, 16%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

94
Total Citations
View Citations
881
Total Downloads

Downloads (Last 12 months)39
Downloads (Last 6 weeks)2

Reflects downloads up to 14 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ye XSakurai KNair NWang K(2024)Machine Learning Techniques for Sensor-Based Human Activity Recognition with Data Heterogeneity—A ReviewSensors10.3390/s2424797524:24(7975)Online publication date: 13-Dec-2024
https://doi.org/10.3390/s24247975
Le HLakshminarayanan RLi JMishra VIntille S(2024)Collecting Self-reported Physical Activity and Posture Data Using Audio-based Ecological Momentary AssessmentProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36785848:3(1-35)Online publication date: 9-Sep-2024
https://dl.acm.org/doi/10.1145/3678584
Yin YXie LJiang ZXiao FCao JLu S(2024)A Systematic Review of Human Activity Recognition Based on Mobile Devices: Overview, Progress and TrendsIEEE Communications Surveys & Tutorials10.1109/COMST.2024.335759126:2(890-929)Online publication date: Oct-2025
https://doi.org/10.1109/COMST.2024.3357591
Al-Saad MRamaswamy LBhandarkar S(2024)JS-Siamese: Generalized Zero Shot Learning for IMU-based Human Activity RecognitionPattern Recognition10.1007/978-3-031-78354-8_26(407-424)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/978-3-031-78354-8_26
Guo XWang YCheng JChen YGuo XWang YCheng JChen Y(2024)Fitness Assistance Using Motion SensorMobile Technologies for Smart Healthcare System Design10.1007/978-3-031-57345-3_6(127-152)Online publication date: 3-Jul-2024
https://doi.org/10.1007/978-3-031-57345-3_6
Guo XWang YCheng JChen YGuo XWang YCheng JChen Y(2024)Personalized Fitness Assistance Using Commodity WiFiMobile Technologies for Smart Healthcare System Design10.1007/978-3-031-57345-3_3(49-82)Online publication date: 3-Jul-2024
https://doi.org/10.1007/978-3-031-57345-3_3
Wang WLi Q(2023)Generalized Zero-Shot Activity Recognition with Embedding-Based MethodACM Transactions on Sensor Networks10.1145/358269019:3(1-25)Online publication date: 5-Apr-2023
https://dl.acm.org/doi/10.1145/3582690
Zhang LZheng DYuan MHan FWu ZLiu MLi X(2023)MultiSense: Cross-labelling and Learning Human Activities Using Multimodal Sensing DataACM Transactions on Sensor Networks10.1145/357826719:3(1-26)Online publication date: 17-Apr-2023
https://dl.acm.org/doi/10.1145/3578267
Fallahzadeh RAshari ZAlinia PGhasemzadeh H(2023)Personalized Activity Recognition Using Partially Available Target DataIEEE Transactions on Mobile Computing10.1109/TMC.2021.307143422:1(374-388)Online publication date: 1-Jan-2023
https://doi.org/10.1109/TMC.2021.3071434
Reining CNair NNiemann FRueda FFink G(2023)A Tutorial on Dataset Creation for Sensor-based Human Activity Recognition2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops)10.1109/PerComWorkshops56833.2023.10150401(453-459)Online publication date: 13-Mar-2023
https://doi.org/10.1109/PerComWorkshops56833.2023.10150401
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents