More Web Proxy on the site http://driver.im/

research-article

Multimedia content creation using societal-scale ubiquitous camera networks and human-centric wearable sensing

Authors:

Mathew Laibowitz,

Joseph A. ParadisoAuthors Info & Claims

MM '10: Proceedings of the 18th ACM international conference on Multimedia

Pages 571 - 580

https://doi.org/10.1145/1873951.1874026

Published: 25 October 2010 Publication History

Abstract

We present a novel approach to the creation of user-generated, documentary video using a distributed network of sensor-enabled video cameras and wearable on-body sensor devices. The wearable sensors are used to identify the subjects in view of the camera system and label the captured video with real-time human-centric social and physical behavioral information. With these labels, massive amounts of continually recorded video can be browsed, searched, and automatically stitched into cohesive multimedia content. This system enables naturally occurring human behavior to drive and control a multimedia content creation system in order to create video output that is understandable, informative, and/or enjoyable to its human audience. The collected sensor data is further utilized to enhance the created multimedia content such as by using the data to edit and/or generate audio score, determine appropriate pacing of edits, and control the length and type of audio and video transitions directly from the content of the captured media. We present the design of the platform, the design of the multimedia content creation application, and the evaluated results from several live runs of the complete system.

References

[1]

Laibowitz, M. Creating Cohesive Video with the Narrative-Informed use of Ubiquitous Wearable and Imaging Sensor Networks. PhD Thesis. January 2010.

[2]

N. W. Gong, M. Laibowitz and J. A. Paradiso. Experiences and Challenges in Deploying Potentially Invasive Sensor Systems for Dynamic Media Applications. Cloud-Mobile Convergence for Virtual Reality Workshop. 2010.

[3]

K. S. J. Pister, J. M. Kahn and B. E. Boser. Smart dust: communicating with a cubic-millimeter computer. Computer, 34(1):44--51, Jan. 2001.

Digital Library

[4]

Steve Hodges, Lyndsay Williams, Emma Berry, Shahram Izadi, James Srinivasan, Alex Butler, Gavin Smyth, Narinder Kapur, Ken Wood. SenseCam: a Retrospective Memory Aid In Dourish and A. Friday (Eds.): Ubicomp 2006, LNCS 4206, pp.177--193, 2006.

Digital Library

[5]

http://www.vicon.com/SenseCam

[6]

Wolf, Ozer, Lv. Smart cameras for embedded systems. In IEEE Computer, volume 35, pages 48--53, September 2002

Digital Library

[7]

Thiago Teixeira, Dimitrios Lymberopoulos, Eugenio Culurciello, Yiannis Aloimonos, Andreas Savvides. A Lightweight Camera Sensor Network Operating on Symbolic Information, Proceedings of First Workshop on Distributed Smart Cameras 2006, ACM SenSys 2006

[8]

Lymberopoulos, D, Teixeira, T, Savvides, A. Macroscopic Human Behavior Interpretations Using Distributed Imagers and Other Sensors, under review for publication in Transactions of IEEE

[9]

Wu-chi Feng, Brian Code, Ed Kaiser, Mike Shea, Wu-chang Feng. Panoptes: scalable low-power video sensor networking technologies. 11th ACM Multimedia, pages 90--91, New York, NY, USA, 2003.

Digital Library

[10]

Gordon Bell, Jim Gemmell. "A Digital Life," Scientific American, 296(3), February 2007, pp. 58--65.

[11]

http://erc.europa.eu/

[12]

T. Choudhury and A. Pentland. Sensing and modeling human networks using the sociometer. In Wearable Computers, 2003. pages 216--222, Oct. 2005.

Digital Library

[13]

Eagle, N and Pentland, A. Reality mining: sensing complex social systems. Personal Ubiquitous Comput., 10(4):255--268, 2006.

Digital Library

[14]

Mathew Laibowitz, Jonathan Gips, Ryan Aylward, Alex Pentland, Joseph A. Paradiso. A sensor network for social dynamics. IPSN 2006. pp. 483--491, 19--21 April 2006.

Digital Library

[15]

Daniel Olguin Olguin, Benjamin N. Waber, Taemie Kim, Akshay Mohan, Koji Ara, and Alex (Sandy) Pentland. Sensible Organizations: Technology and Methodology for Automatically Measuring Organizational Behavior. IEEE Trans. on Systems, Man, and Cybernetics - Part B, 39(1), Feb. 2009, pp. 43--55.

Digital Library

[16]

Singh and Williams. Lifenet: a propositional model of ordinary human activity. In Proceedings at K-CAP 2003, 2003.

[17]

Weld et al. Automatically personalizing user interfaces. In IJCAI03, Acapulco, Mexico, August 2003.

Digital Library

[18]

Guensler R. Wolf J and Bachman W. Elimination of the travel diary: an experiment to derive trip purpose from gps travel data. In Transportation Research, 2001.

[19]

Beth Logan, Jennifer Healey, Matthai Philipose, Emmanuel Tapia, and Stephen Intille. "A long-term evaluation of sensing modalities for activity recognition," in Proceedings of the International Conference on Ubiquitous Computing, vol. LNCS 4717. Springer, 2007, pp. 483--500.

Digital Library

[20]

David Bannach, Oliver Amft, and Paul Lukowicz, "Rapid Prototyping of Activity Recognition Applications," IEEE Pervasive Computing, pp. 22--31, April, 2008

Digital Library

[21]

Duda et al. Pattern Classification (2nd Edition). Wiley-Interscience, November 2000.

Digital Library

[22]

E. Wertheim. Historical background of organizational behavior.

[23]

V. M. Bove and J. Mallett. Collaborative knowledge building by smart sensors. BT Technology Journal, 22(4):45--51, 2004.

Digital Library

[24]

J. Mallett and V. M. Bove. Eye society. In Multimedia and Expo, 2003. ICME '03. pages 6--9 July 2003.

Digital Library

[25]

Reddy, S., Parker, A., Hyman, J., Burke, J., Estrin, D., and Hansen, M. "Image browsing, processing, and clustering for participatory sensing: lessons from a DietSense prototype," Proc. of EmNets 2007, Cork Ireland, pp. 13--17.

Digital Library

[26]

Sayed Ahmad; Rasit Eskicioglu; Peter Graham; "Design and Implementation of a Sensor Network Based Location Determination Service for use in Home Networks,"Mobile Ad hoc and Sensor Systems (MASS), 2006 pp.622--626, Oct. 2006

[27]

Oksana Bulgakowa dir., The Factory of Gestures - Body Language in Film. PPMedia and Stanford Humanities Lab. DVD. 160 minutes. 2007.

[28]

Laibowitz and Paradiso. "The UbER-Badge, A Versatile Platform at the Juncture Between Wearable and Social Computing," in Advances in Pervasive Computing, OCG, 2004, pp.363--368.

[29]

Picard et al. The handwave bluetooth skin conductance sensor. In ACII, volume 3784 of Lecture Notes in Computer Science, pages 699.706. Springer, 2005.

Digital Library

[30]

http://www.youtube.com/watch?v=rDv7D2xeKrs

[31]

http://www.youtube.com/watch?v=vw9outkeR1k

[32]

http://www.youtube.com/watch?v=ME1a0gSvMD8

[33]

http://www.youtube.com/watch?v=Wa4dXm7 yAE

Cited By

Flintham MVelt RWilson MAnstead EBenford SBrown APearce TPrice DSprinks JBegole BKim JInkpen KWoo W(2015)Run Spot RunProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems10.1145/2702123.2702463(747-756)Online publication date: 18-Apr-2015
https://dl.acm.org/doi/10.1145/2702123.2702463
Berry ELu YSu W(2015)Using Global Camera Networks to Create Multimedia ContentProceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)10.1109/CCBD.2015.21(231-234)Online publication date: 4-Nov-2015
https://dl.acm.org/doi/10.1109/CCBD.2015.21
Ishikawa SShinba TKiriyama SKitazawa STakebayashi Y(2012)Method for Creating Content for Understanding Child Behavior for Use as Child-rearing SupportThe Journal of The Institute of Image Information and Television Engineers10.3169/itej.66.J2366:1(J23-J29)Online publication date: 2012
https://doi.org/10.3169/itej.66.J23
Show More Cited By

Index Terms

Multimedia content creation using societal-scale ubiquitous camera networks and human-centric wearable sensing

Recommendations

Semantic annotation of personal video content using an image folksonomy
ICIP'09: Proceedings of the 16th IEEE international conference on Image processing

The increasing popularity of user-generated content (UGC) requires effective annotation techniques in order to facilitate precise content search and retrieval. In this paper, we propose a new approach for the semantic annotation of personal video ...
A Preliminary Exploration of YouTubers' Use of Generative-AI in Content Creation
CHI EA '24: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems

Content creators increasingly utilize generative artificial intelligence (Gen-AI) on platforms such as YouTube, TikTok, Instagram, and various blogging sites to produce imaginative images, AI-generated videos, and articles using Large Language Models (...
Secure content delivery in information-centric networks: design, implementation, and analyses
ICN '13: Proceedings of the 3rd ACM SIGCOMM workshop on Information-centric networking

In this paper, we propose a novel secure content delivery framework, for an information-centric network, which will enable content providers (e.g., Netflix and Youtube) to securely disseminate their content to legitimate users via content distribution ...

Comments

Please enable JavaScript to view thecomments powered by Disqus.

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '10: Proceedings of the 18th ACM international conference on Multimedia

October 2010

1836 pages

ISBN:9781605589336

DOI:10.1145/1873951

General Chairs:
Alberto del Bimbo
University of Florence, Italy
,
Shih-Fu Chang
Columbia University, USA
,
Program Chair:
Arnold Smeulders
University of Amsterdam, NL

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 October 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

MM '10

Sponsor:

SIGMM

MM '10: ACM Multimedia Conference

October 25 - 29, 2010

Firenze, Italy

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
354
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 30 Dec 2024

Other Metrics

View Author Metrics

Citations

Cited By

Flintham MVelt RWilson MAnstead EBenford SBrown APearce TPrice DSprinks JBegole BKim JInkpen KWoo W(2015)Run Spot RunProceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems10.1145/2702123.2702463(747-756)Online publication date: 18-Apr-2015
https://dl.acm.org/doi/10.1145/2702123.2702463
Berry ELu YSu W(2015)Using Global Camera Networks to Create Multimedia ContentProceedings of the 2015 International Conference on Cloud Computing and Big Data (CCBD)10.1109/CCBD.2015.21(231-234)Online publication date: 4-Nov-2015
https://dl.acm.org/doi/10.1109/CCBD.2015.21
Ishikawa SShinba TKiriyama SKitazawa STakebayashi Y(2012)Method for Creating Content for Understanding Child Behavior for Use as Child-rearing SupportThe Journal of The Institute of Image Information and Television Engineers10.3169/itej.66.J2366:1(J23-J29)Online publication date: 2012
https://doi.org/10.3169/itej.66.J23
Gong NLaibowitz MParadiso J(2010)Dynamic privacy management in pervasive sensor networksProceedings of the First international joint conference on Ambient intelligence10.5555/1926743.1926753(96-106)Online publication date: 10-Nov-2010
https://dl.acm.org/doi/10.5555/1926743.1926753
Gong NLaibowitz MParadiso J(2010)Dynamic Privacy Management in Pervasive Sensor NetworksAmbient Intelligence10.1007/978-3-642-16917-5_10(96-106)Online publication date: 2010
https://doi.org/10.1007/978-3-642-16917-5_10

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents