Abstract
Augmented Reality enhances user perception by overlaying real world information with virtual computer-generated information. The aims of the 6 th Sense project are to improve real-time interaction between the real environment and the virtual world and to maximize the user experience in mobile Augmented Reality. To achieve these objectives a generic framework constituted of two main layers is proposed. The End-to-End Adaptation Layer adapts in real-time the parameters of the Augmented Reality system to provide the user with the best possible experience despite the varying operating conditions such as the transmission link and user head motion. The Generic Augmented Reality Layer encompasses solutions to the problem of overlaying adequate information in the real scene and manages multimodal interaction with the virtual environment.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Costanza, E., Fjeld, M.: Augmented Reality, a Survey. In: Human Machine Interaction. LNCS. Springer, Heidelberg (2009)
Walsh, K., Pawlowski, S.: Virtual reality: A technology in need of IS research. Communications of the AIS 8, 297–313 (2002)
Card, S.K., Mackinlay, J.D., Shneiderman, B.: Readings in information visualization: using vision to think. Morgan Kaufmann Publishers, San Francisco (1999)
Holmquist, L.E.: Focus+context visualization with flip zooming and the zoom browser. In: Extended Abstracts on Human Factors in Computing Systems: Looking to the Future (CHI 1997), Atlanta, Georgia, pp. 263–264. ACM, New York (1997)
Spence, R.: Information Visualization: Design for Interaction, 2nd edn. Prentice-Hall, Upper Saddle River (2007)
Kalkusch, M., Lidy, T., Knapp, N., Reitmayr, G., Kaufmann, H., Schmalstieg, D.: Structured visual markers for indoor pathfinding. In: Proceedings of the first International Workshop on Augmented Reality Toolkit (2002)
Bolt, R.A.: “Put-that-there”: Voice and gesture at the graphics interface. In: Proceedings of the 7th Annual Conference on Computer Graphics and Interactive Techniques, Seattle, Washington, United States, pp. 262–270. ACM, New York (1980)
Reithinger, N., Bergweiler, S., Engel, R., Herzog, G., Pfleger, N., Romanelli, M., Sonntag, D.: A look under the hood: design and development of the first smartweb system demonstrator. In: Proceedings of the 7th International Conference on Multimodal Interfaces, Torento, Italyq, pp. 159–166. ACM, New York (2005)
Jaimes, A., Sebe, N.: Multimodal human-computer interaction: A survey. Computer Visioin and Image Understanding 108(1-2), 116–134 (2007)
Wagner, D., Schmalstieg, D., Billinghurst, M.: Handheld AR for Collaborative Edutainment. LNCS, pp. 85–96. Springer, Heidelberg (2006)
Behzadan, A.H., Khoury, H.M., Kamat, V.R.: Structure of an extensible augmented reality framework for visualization of simulated construction processes. In: Proceedings of the 38th Conference on Winter Simulation, Monterey, California, pp. 2055–2062 (2006)
Piekarski, W., Thomas, B.: Using ARToolKit for 3D hand position tracking in mobile outdoor environments. In: Augmented Reality Toolkit, The first IEEE International Workshop (2002)
Gleue, T., Dähne, P.: Design and implementation of a mobile device for outdoor augmented reality in the archeoguide project. In: Proceedings of the Conference on Virtual Reality, Archeology, and Cultural Heritage, Glyfada, Greece, pp. 161–168. ACM, New York (2001)
Cheok, A.D., Yang, X., Ying, Z.Z., Billinghurst, M., Kato, H.: Touch-space: Mixed reality game space based on ubiquitous, tangible, and social computing. Personal Ubiquitous Computing 6(5-6), 430–442 (2002)
Bauer, M., Bruegge, B., Klinker, G., MacWilliams, A., Reicher, T., Riss, S., Sandor, C., Wagner, M.: Design of a component-based augmented reality framework. In: Proceedings of the International Symposium on Augmented Reality, pp. 45–54. ACM and IEEE (2001)
Schmalstieg, D., Fuhrmann, A., Hesina, G., Szalaári, Z., Encarnaçäo, L.M., Gervautz, M., Purgathofer, W.: The studierstube augmented reality project. Presence: Teleoperators and Virtual Environments 11(1), 33–54 (2002)
Henrysson, A., Ollila, M.: UMAR: Ubiquitous mobile augmented reality. In: Proceedings of the 3rd International Conference on Mobile and Ubiquitous Multimedia (MUM 2004), pp. 41–45. ACM, New York (2004)
Wagner, D., Schmalstieg, D.: Handheld augmented reality displays. In: Proceedings of the Virtual Reality Conference, pp. 35–36. IEEE, Los Alamitos (2006)
Peternier, A., Vexo, F., Thalmann, D.: Wearable mixed reality system in less than 1 pound. In: Proceedings of the 12th Eurographics Symposium on Virtual Environment (2006)
Mann, S.: Wearable computing: A first step toward personal imaging. Computer 30(2), 25–32 (1997)
Behringer, R., Tam, C., McGee, J., Sundareswaran, S., Vassiliou, M.: A wearable augmented reality testbed for navigation and ccontrol, built solely with commercial-off-the-shelf (COTS) hardware. In: Proceedings of the International Symposium on Augmented Reality (ISAR 2000), pp. 12–19. IEEE and ACM (2000)
Geiger, C., Kleinnjohann, B., Reimann, C., Stichling, D.: Mobile AR4ALL. In: Proceedings of the International Symposium on Augmented Reality, pp. 181–182. IEEE and ACM (2001)
Pasman, W., Woodward, C.: Implementation of an augmented reality system on a PDA. In: Proceedings of the 2nd International Symposium on Mixed and Augmented Reality, pp. 276–277. IEEE and ACM (2003)
Wagner, D., Schmalstieg, D.: First steps towards handheld augmented reality. In: Proceedings of the 7th International Symposium on Wearable Computers, pp. 127–135. IEEE, Los Alamitos (2003)
Pasman, W., Jansen, F.: Distributed low-latency rendering for mobile AR. In: Proceedings of the International Symposium on Augmented Reality, pp. 107–113. IEEE and ACM (2001)
Jain, R.: Quality of experience. Multimedia, IEEE 11(1), 95–96 (2004)
Prasad, A., Esmailzadeh, R., Winkler, S., Ihara, T., Rohani, B., Pinguet, B., Capel, M.: Perceptual quality measurement and control: Definition, application and performance. In: Proceedings of the 4th International Symposium on Wireless Personal Multimedia Communications, Aalborg, Denmark, vol. 2, p. 255 (2001)
Winkler, S.: Vision models and quality metrics for image processing applications. PhD thesis, Ecole Polytechnique Fédérale de Lausanne, Lausanne (2000)
Sheikh, H.R., Bovik, A.C.: Image information and visual quality. IEEE Transactions on Image Processing 15(2), 430–444 (2006)
Verscheure, O., Garcia, X., Karlsson, G., Hubaux, J.P.: User-oriented QoS in packet video delivery. IEEE Network 12(6), 12–21 (1998)
Ghinea, G., Thomas, J.P., Fish, R.S.: Multimedia, network protocols and users - bridging the gap. In: Proceedings of the 7th International Conference on Multimedia (Part 1) (MULTIMEDIA 1999), pp. 473–476. ACM, New York (1999)
Ghinea, G., Fish, R.S., Thomas, J.P.: Using quality of perception for improved multimedia communication. In: Proceedings of the AFRICON, vol. 2, pp. 1241–1246. IEEE, Los Alamitos (1999)
Takahata, K., Uchida, N., Shibata, Y.: QoS control for real time video stream over hybrid network by wired and wireless LANs. In: Proceedings of the 17th International Conference on Advanced Information Networking and Applications (AINA 2003), pp. 45–51. IEEE, Los Alamitos (2003)
Bai, Y., Ito, M.: Network-level loss control schemes for streaming video. In: Proceedings of the International Conference on Multimedia and Expo (ICME 2004), vol. 1, pp. 495–498. IEEE, Los Alamitos (2004)
Cranley, N.: User-Perceived Quality-Aware Adaptation of Streamed Multimedia over Best-effort IP Networks. PhD thesis, National University of Ireland, Department of Computer Science Faculty of Science University College Dublin Belfield, Dublin 4 (March 2004)
Cranley, N., Perry, P., Murphy, L.: Dynamic content-based adaptation of streamed multimedia. Journal of Network and Computer Applications 30(3), 983–1006 (2007)
Perritaz, D., Salzmann, C., Gillet, D.: User perception model for wearable supervision systems. In: Proceedings of the 4th International Conference on Mobile Technology, Applications and Systems and the 1st International Symposium on Computer Human Interaction in Mobile Technology (Mobility 2007) (September 2007)
Azuma, R.T.: A survey of augmented reality. Presence-Teleoperators and Virtual Environments 6(4), 355–385 (1997)
Allison, R.S., Harris, L.R., Jenkin, M., Jasiobedzka, U., Zacher, J.E.: Tolerance of temporal delay in virtual environments. In: Proceedings of the Virtual Reality Conference (VR 2001), pp. 247–254. IEEE, Los Alamitos (2001)
Ellis, S.R., Young, M.J., Adelstein, B.D., Ehrlich, S.M.: Discrimination of changes in latency during head movement. In: Proceedings of the 8th International Conference on Human-Computer Interaction (HCI 1999) on Human-Computer Interaction: Communication, Cooperation, and Application Design, vol. 2, pp. 1129–1133. ACM, New York (1999)
Adelstein, B.D., Lee, T.G., Ellis, S.R.: Head tracking latency in virtual environments: Psychophysics and a model. In: Proceedings of the 47th Annual Meeting on Human Factors and Ergonomics Society, pp. 2083–2087 (2003)
Barfield, W., Hendrix, C.: The effect of update rate on the sense of presence within virtual environments. Virtual Reality 1(1), 3–15 (1995)
Lai, W.Y., Duh, H.B.L.: Effects of frame eate for visualization of dynamic quantitative information in a head-mounted display. In: Proceedings of the International Conference on Systems, Man and Cybernetics, vol. 7, pp. 6485–6490. IEEE, Los Alamitos (2004)
Smets, G.J., Overbeeke, K.J.: Trade-off between resolution and interactivity in spatial task performance. IEEE Computer Graphics and Applications 15(5), 46–51 (1995)
Ryu, J., Hashimoto, N., Sato, M.: Influence of resolution degradation on distance estimation in virtual space displaying static and dynamic image. In: Proceedings of the International Conference on Cyberworlds (CW 2005). IEEE, Los Alamitos (2005)
Alben, L.: Quality of experience: Defining the criteria for effective interaction design. Interactions 3(3), 11–15 (1996)
Sielhorst, T., Sa, W., Khamene, A., Sauer, F., Navab, N.: Measurement of absolute latency for video see through augmented reality. In: Proceedings of the 6th International Symposium on Mixed and Augmented Reality (ISMAR 2007), pp. 215–220. IEEE and ACM (2007)
Saw, Y.S.: Rate-quality optimized video coding. Kluwer Academic Publishers, Norwell (1999)
Chen, Z., Ngan, K.N.: Recent advances in rate control for video coding. Signal Processing: Image Communications Magazine 22(1), 19–38 (2007)
Yu, S., Ahmad, I.: New rate control algorithm for MPEG-4 video coding. In: Proceedings of the Conference on Visual Communications and Image Processing. The Society of Photo-Optical Instrumentation Engineers (SPIE) Conference, vol. 4671, pp. 698–709 (January 2002)
Wong, C.W., Au, O.C., Lam, H.K.: PID-based real-time rate control. In: Proceedings of the International Conference on Multimedia and Expo (ICME 2004), vol. 1, pp. 221–224. IEEE, Los Alamitos (2004)
Shen, L., Liu, Z., Zhang, Z., Shi, X.: Rate control based on an incremental proportional-integral-differential algorithm. Optical Engineering 46(7), 077002–1–7 (2007)
Azuma, R.T.: Predictive Tracking for Augmented Reality. PhD thesis, University of North Carolina at Chapel Hill (February 1995)
Tumanov, A., Allison, R., Stuerzlinger, W.: Variability-aware latency amelioration in distributed environments. In: Proceedings of the Virtual Reality Conference (VR 2007), pp. 123–130. IEEE, Los Alamitos (2007)
Chakareski, J., Frossard, P.: Adaptive systems for improved media streaming experience. IEEE Communications Magazine 45(1), 77–83 (2007)
Cranley, N., Murphy, L., Perry, P.: Perceptual quality adaptation (PQA) algorithm for 3GP and multi-tracker MPEG-4 content over wireless IP networks. In: Proceedings of the 5th International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC 2004), vol. 3, pp. 2107–2112. IEEE, Los Alamitos (2004)
Muntean, G.M., Cranley, N.: Resource efficient quality-oriented wireless broadcasting of adaptive multimedia content. IEEE Transactions on Broadcasting 53(1), 362–368 (2007)
Kim, T., Ammar, M.H.: Optimal quality adaptation for scalable encoded video. IEEE Journal on Selected Areas in Communications 23(2), 344–356 (2005)
Ni, Z.F., Chen, Z.Z., Ngan, K.N.: A real-time video transport system for the best-effort Internet. Signal Processing-Image Communication 20(3), 277–293 (2005)
Salzmann, C., Gillet, D., Müllhaupt, P.: End-to-end adaptation scheme for ubiquitous remote experimentation. Personal and Ubiquitous Computing (October 2007)
Munshi, A., Ginsburg, D., Shreiner, D.: OpenGL ES 2.0 programming guide. Addison-Wesley Professional, Reading (2008)
Grasset, R., Boissieux, L., Gascuel, J.D., Schmalstieg, D.: Interactive mediated reality. In: Proceedings of the 6th Australasian User Interface Conference (AUIC 2005), Newcastle, Australia (February 2005)
Ozuysal, M., Fua, P., Lepetit, V.: Fast keypoint recognition in ten lines of code. In: Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR 2007), pp. 1–8. IEEE, Los Alamitos (2007)
Stock, C., Siegl, H., Zinsser, T., Pinz, A.: Hybrid tracking for collaborative mobile AR. In: Deliverable D.4.4, Graz University of Technology (May 31, 2004)
State, A., Hirota, G., Chen, D.T., Garrett, W.F., Livingston, M.A.: Superior augmented reality registration by integrating landmark tracking and magnetic tracking. In: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pp. 429–438. ACM, New York (1996)
Rolland, J.P., Baillot, Y., Goon, A.A.: A survey of tracking technology for virtual environments. Lawrence Erlbaum Associates, Orlando (1995)
Wormell, D., Foxlin, E., Katzman, P.: Advanced inertial-optical tracking system for wide area mixed and augmented reality systems. In: Proceedings of the 10th International Immersive Projection Technologies Workshop (2007)
Wagner, D., Schmalstieg, D.: ARToolKitPlus for pose tracking on mobile devices. In: Computer Vision Winter Workshop, pp. 6–8 (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Perritaz, D. et al. (2009). 6 th Sense– Toward a Generic Framework for End-to-End Adaptive Wearable Augmented Reality. In: Lalanne, D., Kohlas, J. (eds) Human Machine Interaction. Lecture Notes in Computer Science, vol 5440. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-00437-7_11
Download citation
DOI: https://doi.org/10.1007/978-3-642-00437-7_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-00436-0
Online ISBN: 978-3-642-00437-7
eBook Packages: Computer ScienceComputer Science (R0)